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1.0  Background 


Most  large-scale  force  level  simulations  assume  perfect  communications.  This  can  lead  to 
significant  limitations  in  the  results  obtained  from  running  these  simulations.  The  vision  of  GIE 
is  to  move,  process,  manage,  and  protect  the  Command  and  Control  Intelligence  Surveillance 
and  Reconnaissance  (C2ISR)  infonnation  that  supports  the  functions  of  Global  Awareness  and 
Dynamic  Planning  and  Execution.  The  mission  of  GIE  is  to  link  aerospace  assets  in-theater  and 
globally,  to  integrate  C2  &  ISR  networks,  to  defend  critical  information  systems  from  cyber 
attack,  and  to  develop  new  information  processing  and  management  techniques.  This  implies  the 
ability  to  construct  large-scale  simulation  environments  for  these  large  force-level  simulations 
that  take  into  account  the  highly  dynamic  nature  of  combat  networks  that  can  include  reach-back 
networks,  layers  of  radio  links  and  networks,  and  a  host  of  communications  vehicles  including 
satellites,  manned  and  unmanned  air  platforms,  ground  and  sea  vehicles  down  to  individual 
soldiers  and  sensor  systems.  The  GIESim  effort  plans  to  fill  the  gaps  in  communications 
modeling,  and  plans  to  accomplish  its  goals  by  assembling  complex,  heterogeneous  simulations 
into  the  appropriate  multi-simulation  environments  required  to  model  the  GIE.  The  General 
Simulation  System  (GSS),  and  the  many  models  and  simulations  built  with  it  have  been  chosen 
as  one  platform  to  take  part  in  the  development  of  the  GIESim/JSB-RD  software  merger. 

In  FY  2004,  the  GIESim  AFRL/IFGC  leadership  team  set  the  goal  of  expanding  on  and  drawing 
upon  the  expertise  and  lessons  learned  in  building  the  DTIG  Multi-Simulation  Demonstration 
(DMSD)  built  for  the  2003  SAB  Review.  In  FY04,  the  GIESim  JTIDS  Simulation  capabilities 
from  PSI  were  merged  into  the  Joint  Semi-Automated  Forces  (JSAF)  simulation  in  AFRL  Rome 
in  conjunction  with  the  JSB-RD  team.  The  central  themes  for  the  FY04  GIESim/JSB-RD 
merger  were: 

•  The  addition  of  communications  modeling  capabilities  to  JSAF  by  interfacing  and  merging 
GIESim  JTIDS  modeling  capabilities.  JSAF  is  a  JFCOM  program  that  is  used  extensively 
for  war  gaming  and  large,  man-in-the  loop  exercises.  However,  JSAF  does  not  include  any 
communications  modeling,  and  simply  assumed  that  communications  always  worked. 

•  Faster  and  Easier  design,  development  and  execution  of  GIESim  simulations. 

•  Tools  and  Technologies  for  GIESim. 

The  focus  of  the  FY05  program  was  to  further  develop  the  GIESim/JSB-RD  merger  by  building 
a  robust  demonstration  of  JSAF-GIESim  interoperability,  and  to  explore  further  experimentation 
with  respect  to  scalability  associated  with  handling  larger  scenario  sizes.  This  report  covers  the 
experimentation  performed  by  PSI  on  the  GIESim  JTIDS  simulation  under  controlled  loads. 
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2.0  Introduction 


This  report  covers  the  experimentation  that  was  performed  by  PSI  to  determine  loading 
capacities  and  scalability  associated  with  the  GIESim  JTIDS  simulation  for  handling  larger 
scenario  sizes  and  networks  of  different  complexity.  Figure  1  illustrates  the  operation  of  the 
GIESim  JTIDS  simulation  with  JSAF.  JSAF  provides  platform  position  updates  to  JTIDS,  and 
makes  transmission  requests  for  specific  platforms.  If  the  transmission  is  successful,  then  JTIDS 
provides  a  response  to  JSAF  that  indicates  the  receiving  platform  and  JSAF  message  ID.  Both 
systems  use  the  same  force  composition  of  Link- 16  platforms,  and  the  JTIDS  simulation  uses  a 
network  design  that  was  developed  for  the  operations  within  JSAF.  Details  on  the  overall  system 
are  provided  in  references  [1][2]. 


Figure  1  -  GIESim  JTIDS  Simulation  Interface  to  JSAF 


The  GIESim  JTIDS  Simulation  was  derived  from  a  variation  of  the  PSI  Link- 16  Network 
Management  System  (NMS)  as  shown  in  Figure  2.  The  Link-16/JTIDS  Planning  Tool  is  used  to 
capture  and  refine  network  requirements  for  the  dynamic  operations  planned  for  the  force 
composition  in  the  Scenario.  The  Planning  Tool  can  also  be  used  to  create  scenarios  that  consist 
of  Link- 16  platforms  and  motion  paths.  The  Planning  Tool  can  test  both  RF  JTIDS  connectivity 
and  Link- 16  network  connectivity  while  designing  the  scenarios  and  networks.  The  Link- 16 
NMS  was  used  to  design  the  initial  interoperability  test  and  demo  scenario  used  with  JSAF.  This 
scenario,  which  was  dubbed  the  “Wow”  scenario,  is  shown  in  Figure  3.  The  goal  of 
experimentation  was  to  explore  larger  scenarios  and  more  complex  networks  to  determine  the 
performance  “envelop”  of  the  JTIDS  simulation. 
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Figure  2  -  PSI  Link-16  Network  Management  System 
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Figure  3  -  Initial  Scenario  developed  for  testing  and  demo  with  JSAF 
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3.0  Design  of  Experiment 


This  section  of  the  report  covers  the  overall  design  of  experiment  for  load  testing  the  GIESim 
JTIDS  simulation,  and  reviews  the  considerations,  challenges,  tools  and  ultimately  the  details  of 
the  experiments  that  were  designed  and  executed. 

3.1  Scope  of  Challenges 

Experimentation  turned  out  to  be  a  much  more  complex  and  challenging  undertaking  than 
expected.  To  a  large  extent,  this  was  due  to  the  high  number  of  experimental  variables  and  the 
potentially  large  test  space  that  could  be  explored.  Furthermore,  after  some  initial  considerations 
and  experimental  runs,  the  concept  of  “load  testing”  became  much  more  profound  and  deeper 
than  originally  anticipated.  To  some  extent  this  was  due  to  the  fact  that  platfonn  position 
updates  and  transmission  requests  were  arriving  in  “real  time”,  while  the  simulation  was  trying 
to  maintain  real  time  processing.  This  is  described  in  more  detail  shortly.  Also,  we  identified 
the  need  to  “correlate”  external  driving  factors  with  the  size  and  complexity  of  the  scenario  and 
network  design  and  internally  obtained  metrics  that  provide  an  assessment  of  simulation  load. 

Figure  4  illustrates  most  of  the  load  factors  on  the  JTIDS  simulation.  Platform  position  updates 
arrive  with  a  certain  rate,  and  updates  may  be  distributed  over  time  or  come  as  a  group. 
Furthermore,  updates  may  only  come  for  platforms  with  changed  positions,  or  for  all  platforms. 


Figure  4  -  Load  Factors  on  the  GIESim  JTIDS  Simulation 


Load  on  the  system  depends  on  the  frequency  of  transmission  requests,  and  on  the  size  and  types 
of  the  operational  nets  being  requested.  Furthermore,  higher  loads  are  incurred  for  transmissions 
through  networks  with  platforms  that  move  more  rapidly,  since  platform  motion  causes  more 
frequent,  and  costly,  propagation  calculations.  Also  the  number  of  destinations  and  relays  in 
each  operational  net  also  influences  loading.  Generally  nets  with  larger  numbers  of  destinations 
introduce  higher  loads.  Relays  in  particular  can  dramatically  impact  performance. 
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We  determined  that  we  needed  to  “instrument”  the  JTIDS  simulation  to  extract  internal  measures 
of  performance  that  could  be  correlated  to  the  external  driving  factors.  These  internal  measures, 
combined  with  the  driving  factors,  would  allow  us  to  characterize  the  “load”  on  the  system. 

Also,  a  controlled  means  of  sending  different  volumes  of  position  updates  and  transmission 
requests  was  required  to  obtain  repeatable  data.  Reception  responses  from  JTIDS  also  had  to  be 
captured  for  later  analysis  and  archiving. 

We  also  considered  the  potential  impact  of  HLA  overhead  on  “real  time”  perfonnance.  We  also 
wanted  to  compare  performance  on  computers  with  different  capabilities  to  assess  sensitivity  to 
PC  parameters. 

The  subsections  that  follow  provide  deeper  descriptions  of  some  of  the  challenges  that  were 
considered  and  how  we  addressed  them. 


3.1.1  What  is  Time? 

The  title  of  this  section  may  seem  frivolous,  however  the  question  of  what  time  is  when  applied 
to  a  simulation  environment  is  very  important  to  consider  and  highly  relevant  to  this  application 
of  JTIDS  with  JSAF.  JSAF  and  JTIDS  are  planned  to  run  in  “real  time”,  which  means  that  both 
simulation  JSAF  and  JTIDS  clocks  are  “constrained”  to  the  system  clock  that  each  is  running  on. 
This  implies  that  ideally  one  second  on  each  system  will  be  the  same  as  one  second  on  the  wall 
clock. 

What  this  means  is  that  each  simulation  will  attempt  to  schedule  events  as  they  should  occur 
according  to  the  real-time  (wall)  clock.  However,  in  any  simulation,  a  certain  amount  of  time 
slip  will  occur  between  the  simulation  clock  and  the  real  time  system  clock.  For  instance, 
suppose  that  two  events  are  scheduled  at  the  same  time.  The  first  event  will  take  a  finite  amount 
of  time  to  process.  This  implies  that  the  second  event  will  get  processed  later  than  scheduled, 
which  is  a  time  slip.  Figure  5  illustrates  this.  When  (in  this  case)  GSS  looks  at  a  scheduled 
process,  if  the  current  real-time  is  later  than  the  scheduled  event  time,  e.g.,  a  time  slip  has 
occurred,  then  the  process  is  run  immediately  as  shown  for  P3  in  Figure  5.  On  the  other  hand,  if 
the  scheduled  event  time  is  a  “future  time”,  then  the  simulation  time  will  catch  up  to  real  time  as 
shown  for  P4.  If  the  average  time  interval  between  events,  e.g.,  the  event  rate,  is  shorter  than  the 
time  required  to  process  events  then  the  simulation  clock  will,  on  average,  be  synchronized  with 
the  real  time  system  clock. 

A  comparison  measure  of  the  simulation  time  to  real  time  is  the  Sim/Real  ratio.  Figure  6 
illustrates  how  increasing  message  transmission  request  rates  can  impact  time  slip  of  the 
simulation  clock,  and  lower  the  Sim/Real  ratio.  At  some  point,  the  Sim/Real  ratio  becomes  too 
low.  The  question  is  then:  “What  is  too  low?”  The  experiments  perfonned  under  this  task  were 
aimed  at  answering  this  question,  and  to  detennine  a  rough  “perfonnance  envelop”  for  JTIDS. 
From  the  internal  perspective  of  the  simulation  receiving  requests  in  real  time,  as  the  simulation 
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clock  slows  down,  the  “apparent”  rate  of  requests  measured  against  the  simulation  clock 
increases.  This  implies  that  requests  pile  up  within  the  simulation  and  can  cause  the  simulation 
clock  to  stay  behind  for  a  longer  duration. 


Real 
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At  same  time 
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Clock 


Figure  5  -  Comparison  of  Simulation  Clock  to  Real  Time  during  processing 
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Figure  6  -  Conceptual  Effect  of  Message  Request  Rate  on  Simulation  Time  Slip 
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3.1.2  Latency  Variations  and  Accuracy 

Each  Link- 16  Operational  Net  has  a  specific  response  time  that  is  defined  at  design  time. 
Therefore  on  average,  responses  to  transmission  requests  should  ideally  have  latencies  that  are 
do  not  exceed  the  response  time  of  each  network.  The  key  term  is  “on  average’.  The  response 
time  of  a  net  is  determined  by  how  many  time  slots  are  assigned  to  it  for  transmission.  Figure  7 
shows  the  1536  time  slots  in  JTIDS  that  repeat  every  12  seconds.  This  repetition  rate  is  referred 
to  as  the  Repetition  Rate  Number  (RRN).  A  message  that  is  queued  for  transmission  can  be 
transmitted  whenever  its  assigned  time  slot  comes  around.  Figure  7  depicts  two  different  cases 
for  transmission  requests.  Transmit  Request  A  comes  in  a  few  time  slots  before  its  next 
allocated  time  slot  (red),  and  therefore  the  message  will  have  a  short  latency.  Transmit  Request 
B,  however,  arrives  many  time  slots  before  it  can  transmit  (green),  which  means  that  its  message 
latency  will  be  much  longer.  When  messages  arrive  “just”  before  the  assigned  time  slot,  they 
will  have  near-zero  latency,  whereas  messages  that  arrive  just  after  the  assigned  time  slot  will 
incur  the  worst-case  latency. 


Transmit 


Figure  7  -  Transmit  Requests  vs.  Scheduled  Receive  Times  for  JTIDS 

The  time-slot  nature  of  JTIDS  communications  can  introduce  broad  variations  in  the  response 
time  of  transmission.  Nets  with  very  high  response  times,  or  high  throughputs,  can  be  allocated 
many  time  slots  so  that  transmit  opportunities  occur  frequently. 

In  the  JTIDS  simulation,  latency  is  computed  from  the  time  the  message  request  comes  in  to  the 
time  that  the  message  is  received  within  the  simulation.  Because  of  the  desire  to  run  in  real-time, 
latency  is  now  computed  in  terms  of  wall  clock  (system)  time.  As  the  simulation  becomes  more 
and  more  loaded,  the  accuracy  of  the  computed  latency  will  drop. 
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3. 1.2.1  Transmission  Requests,  Bus  Delays,  and  Scheduling 

In  Link- 16  radios,  other  factors  affect  latency.  The  bus  between  the  Link- 16  Host  and  the  Link- 
16  terminal  can  introduce  additional  delays.  In  the  JTIDS  simulation,  a  normal  distribution  is 
used  to  randomly  introduce  50  to  150  ms  of  delay.  Based  on  our  analysis,  bus  delays  make 
negligible  contributions  to  message  latency. 

The  point  at  which  the  HOST  OUTBOUND  QUEUE  model  schedules  the  bus  delay  is  the  first 
schedule  statement  that  occurs  following  a  transmission  request.  The  significance  of  this  will 
become  clear  shortly.  See  Figure  8  for  reference. 

In  the  JTIDS  simulation  HLA  events  including  transmission  requests  are  accumulated  in  near 
real  time.  Once  the  transmission  request  has  been  scheduled  the  simulation  clock  determines 
when  the  request  is  actually  honored. 
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When  the  simulation  is  lightly  loaded,  i.e.  the  Sim/Real  ratio  is  very  close  to  1.0,  the 
transmissions  will  occur  in  the  desired  (wall  clock)  time.  However,  as  the  simulation  gets  more 
and  more  loaded,  then  the  simulation  queue  size  will  increase  and  the  transmission  requests  may 
get  queued  up  behind  many  other  events  that  are  running  late  relative  to  wall  time.  The 
consequence  is  that  transmission  latency  through  the  system  will  be  longer  than  expected. 
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3. 1.2.2  Transmission  Requests  and  Net  Overdrive 

In  the  JTIDS  simulation  each  net  has  a  transmit  buffer  in  the  JTIDS  Processor  Model  that  can 
hold  messages  for  transmission.  Since  any  net  can  only  send  messages  at  a  certain  rate,  the 
buffer  is  intended  to  temporarily  hold  messages. 

If  transmission  requests  arrive  at  a  steady  rate  that  is  faster  than  the  net  can  handle,  then 
eventually  the  input  buffer  will  overflow  and  excess  requests  will  be  lost.  This  can  happen  even 
if  the  system  is  lightly  loaded,  and  is  a  case  of  over  driving  the  net. 

Another  consequence  of  the  transmit  buffers  is  that  short  bursts  of  transmission  requests  on  one 
net  may  become  deeply  buffered.  As  a  result,  the  “latency”  of  the  last  request  loaded  into  the 
transmit  buffer  will  be  much  longer  than  the  first  request  in  the  buffer  since  all  proceeding 
messages  in  the  buffer  must  be  sent  before  the  final  message  is  transmitted. 

Since  brief  bursts  of  requests  are  “stored”  in  the  transmit  buffer  and  are  metered  out  at  the  rate 
supported  by  the  net,  the  apparent  load  on  the  system  will  not  increase  appreciably  since 
messages  leave  the  buffer  at  a  fixed  rate.  However,  as  noted  earlier,  the  latency  of  the  message 
in  the  buffer  will  suffer. 

As  the  simulation  becomes  loaded,  then  the  real  time  to  handle  messages  starts  to  drop. 
Individual  transmissions  will  take  longer  by  a  factor  that  is  inversely  proportional  to  the 
Sim/Real  ratio.  Latency  of  buffered  transmit  messages  will  therefore  take  substantially  longer. 

Generally,  the  rate  of  transmission  requests  on  any  net  should  be  less  than  the  capacity  of  the  net. 
The  net  capacity  is  determined  by  either  the  specified  response  time  or  the  throughput  required 
for  the  net  -  whichever  is  higher.  In  cases  where  there  are  insufficient  time  slots  to  satisfy  the 
required  capacity,  the  actual  capacity  allocated  will  be  less.  This  can  be  detennined  by 
reviewing  the  time  slots  allocated  to  the  net  with  the  Link- 16  Planning  Tool. 
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3.1.3  Position  Accuracy 

As  a  result  of  frequent  HLA  event  checking,  platform  position  updates  are  handled  in  near  real 
time.  The  new  positions  of  platforms  are  updated  in  the  platform  database  and  the  platform 
icons  are  updated  on  the  graphics  display  as  they  occur.  However,  message  transmission 
requests  may  not  be  handled  in  real  time  depending  on  the  load  on  the  JTIDS  system  at  the  time. 
This  is  a  generic  problem  for  simulations. 


Figure  9  -  Platform  Position  Accuracy 


With  higher  system  loads,  transmission  requests  can  become  queued  in  internal  buffers  including 
the  simulation  queue.  Transmission  requests  are  associated  with  the  platform  positions  at  the 
time  of  the  request,  and  the  impacts  of  system  delays  in  transmissions  vary  in  terms  of  platform 
speed.  This  situation  is  illustrated  in  Figure  9.  Fast  moving  platforms  will  be  out  of  position 
much  quicker  than  slow  moving  or  stationary  platforms  at  the  time  the  message  transmission 
actually  takes  place  after  experiencing  system  delays.  Therefore,  the  physical  platform  positions 
at  the  time  of  message  transmissions  can  lead  to  inaccurate  results.  For  instance,  at  the  time  of  a 
transmission  request  on  a  particular  net,  platforms  in  the  net  may  be  clear  of  terrain  obstructions. 
However,  at  the  time  the  message  goes  through,  the  terrain  might  block  RF  propagation  and  give 
a  false  answer.  All  of  this  of  course  assumes  that  the  extended  latency  incurred  by  system  delays 
is  in  itself  acceptable. 


The  root  of  the  overall  problem  is  time;  real  time  compared  to  simulation  time.  This  implies  that 
during  driven  operations  of  JTIDS  that  the  state  of  the  Sim/Real  ratio  is  monitored  throughout 
the  load  runs.  Our  experimentation  analysis  places  reasonable  limits  on  JTIDS  loading. 
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3.2  Factors  Affecting  JTIDS  Loading 


The  overarching  goal  of  experimentation  is  to  detennine  the  operational  performance  envelop  for 
JTIDS  under  varying  load  conditions  and  sensitivity  to  scaling  of  scenario  sizes  and  networks. 
While  contemplating  our  experiments  and  during  analysis  of  data  we  explored  different  factors 
that  could  influence  loading  of  the  JTIDS  simulation.  Some  loading  factors  have  already  been 
discussed.  This  section  presents  additional  loading  factors  and  provides  some  detailed  insights 
into  each  that  must  be  considered  when  planning  operational  runs  with  JSAF  or  other  large 
forces  simulations  that  might  drive  JTIDS  and  employ  it  to  provide  network  connectivity. 

Figure  10  depicts  some  of  the  load  factors  that  can  affect  loading  of  GIESim  JTIDS.  The  left 
side  of  Figure  10  involves  both  interface  overhead  and  computational  overhead  associated  with 
transmission  processing.  The  right  side  of  Figure  10  shows  the  scenario  and  networks  that  can 
be  loaded  into  JTIDS.  The  magnitude  of  the  computations  required  to  support  transmission 
requests  can  be  directly  affected  by  the  scenario  size  and  complexity  and  size  of  the  Link- 16 
networks  that  are  loaded. 
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Figure  10  -  JTIDS  Internal  Load  Factors 


Sections  3.2.1  through  3.2.4  provide  a  brief  overview  of  some  of  these  factors  and  their 
relationships. 
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3.2.1  Propagation  Calculations  and  Position  Updates 

Propagation  calculations  are  amongst  the  heaviest  computational  loads  in  the  JTIDS  simulation 
even  with  the  PSI  Fast  Propagation  Prediction  System  (FPPS).  JTIDS  keeps  track  of  platfonn 
position  locations  and  avoids  the  costly  propagation  calculation  for  links  between  pairs  of 
platforms  that  have  not  changed  position  since  the  last  calculation.  Since  position  updates  can 
change  platfonn  positions  between  transmission  requests,  updates  can  cause  additional  calls  to 
FPPS.  Consider  the  two  cases  illustrated  in  Figure  below.  Both  cases  assume  that  T1  is  just 
after  a  transmission  has  been  handled. 


Figure  11  -  Propagation  Calculations  for  Moving  Platforms 


In  the  top  case,  platforms  D1  and  D2  are  stationary,  the  source  platfonn  (Src)  is  also  stationary, 
and  D3  and  D4  are  moving.  When  a  transmission  request  comes  in  at  T2,  new  FPPS  calculations 
are  required  between  the  Src  and  D3  and  D4  but  not  with  respect  to  D1  or  D2. 

In  the  lower  case,  the  source  platform  is  moving  in  addition  to  D3  and  D4.  Now  when  a 
transmission  request  comes  in  at  T2,  FPPS  calculations  are  required  for  all  links.  Fast  moving 
platforms  will  change  positions  more  frequently  with  position  updates  than  slow  moving 
platforms.  Airborne  transmitters  will  typically  introduce  a  higher  propagation  load  than  other 
platforms. 
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3.2.2  Size  of  Operational  Nets 


The  size  of  operational  nets  can  directly  influence  computational  load.  Figure  illustrates  several 
operational  nets  of  increasing  size.  When  a  transmission  is  requested  from  the  source  of  an 
operational  net,  the  JTIDS  simulation  builds  a  message  for  transmission  through  the  simulation. 
The  message  that  is  built  contains  a  list  of  destinations,  network  characteristics,  and  other  data 
including  the  payload  required  for  JSAF.  The  completed  message  is  then  moved  to  the  buffer  of 
each  destination  receiver  and  each  destination  is  scheduled  to  receive  the  message  at  the  next 
time  slot  available  to  the  net.  Message  movement  and  scheduling  take  a  finite  amount  of  time. 
Larger  nets  therefore  require  more  time  for  this  step  in  the  transmission  process  to  complete. 


Figure  12  -  Size  of  Operational  Nets 


With  JTIDS  radios,  many  radios  can  be  transmitting  at  once  on  the  same  time  slot.  This  can  be 
due  to  the  use  of  different  J-Nets  (frequency  hopping  patterns),  or  use  of  Contention  Access  or 
be  due  to  net  designs  in  which  transmitters  are  expected  to  be  far  enough  apart.  Figure  13 
illustrates  the  case  where  two  transmitters  are  transmitting  in  the  same  time  slot. 
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Assume  that  the  JTIDS  Host  Model  gets  back-to-back  transmission  requests  with  T2  following 
T 1 .  T 1  moves  its  transmitted  message  to  time  slot  buffer  N  and  then  schedules  destinations  D 1 
and  D2  to  be  received  at  time  T.  T2  then  moves  the  transmit  message  onto  the  buffer  for  time 
slot  N. 


Figure  13  -  Message  Transmission  and  Mutual  Interference 


At  the  scheduled  time,  first  D1  and  then  D2  check  to  see  what  message  they  should  be  receiving 
from  time  slot  N.  Each  in  turn,  does  a  mutual  interference  calculation  to  detennine  if  the  SNR  is 
sufficient  for  it  to  receive.  This  involves  checking  distance  to  the  transmitters  and  propagation 
calculations  to  determine  which  transmitter  has  the  stronger  signal.  If  T1  has  the  stronger  signal 
and  the  SNR  is  sufficient  then  the  message  Ml  is  received,  otherwise  it  is  dropped.  After  this, 

D2  then  performs  a  similar  calculation.  When  a  message  is  received,  then  it  is  passed  to  the  Host 
model,  which  determines  if  the  JSAF  message  was  intended  for  this  particular  receiver.  If  so, 
then  an  HLA  response  message  is  created  that  includes  message  latency  through  JTIDS  and  the 
message  is  send  out  over  HLA. 

Next  Dl,  D2  and  D3  repeat  this  sequence  of  calculations  to  determine  if  message  M2  transmitted 
from  T2  can  be  received  and  how  it  should  be  handled  when  received. 

Obviously  nets  with  larger  numbers  of  destinations  will  incur  higher  processing  overhead.  As 
discussed  in  an  earlier  section,  relative  motion  between  platforms  in  a  net  can  dramatically 
increase  processing  time  due  to  higher  number  of  propagation  calculations  that  are  required. 

Internal  to  the  JTIDS  simulation  a  value  called  “Expected”  is  incremented  each  time  a 
transmission  occurs  by  the  number  of  destinations  in  the  net  being  used.  Expected  is  therefore 
an  internal  measure  of  loading  on  JTIDS. 
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3.2.3  Airborne  Relays 

Many  operational  nets,  particularly  larger  nets,  require  relays  to  ensure  that  all  destinations  can 
be  reached.  Frequently  airborne  relays  are  the  first  choice,  particularly  to  reach  ground 
positions,  because  of  their  altitudes.  The  use  of  relays  complicates  loading  and  makes  load 
assessment  of  nets  inherently  less  predictable. 

Figure  illustrates  operation  with  a  single  relay  Rl.  The  source  platform  transmits  message  ml 
that  is  received  at  destinations  D1  and  D2  and  by  Rl.  However,  destination  D3  is  too  far  from 
the  source  to  receive  the  message.  In  the  next  set  of  adjacent  time  slots,  Rl  will  transmit 
message  ml.  Assuming  that  D3  is  in  range,  D3  will  receive  message  ml .  Note  that  the  figure 
shows  the  source  and  D1  and  D2  dropping  message  ml  from  the  relay.  This  is  because  each 
receiver  keeps  a  list  of  message  IDs  that  have  already  been  received.  FPPS  calculations  are 
performed  to  determine  if  each  receiver  has  an  SNR  value  for  it  to  receive. 


Multiple  relays  in  a  net  make  the  situation  more  complicated  from  the  perspective  of  loading  and 
with  respect  to  understanding  what  happens  at  any  point  in  time.  Figure  14  illustrates  the  case 
for  an  operational  net  with  two  relays.  The  configuration  assumes  that  destinations  D3  and  D4 
and  relay  R2  are  too  far  from  the  source  to  hear  it  directly.  The  different  colored  links  are 
intended  to  represent  the  sequence  in  time  of  transmissions.  Black  represents  the  initial 
transmission  from  the  source.  Red  lines  indicate  the  transmissions  from  the  first  relay  -  Rl . 
Green  lines  indicate  transmissions  from  relay  R2. 

First  Src  transmits  message  ml  to  the  destinations  on  the  net.  Here  we  assume  that  Dl,  D2  and 
Rl  receive  the  message.  Rl  then  retransmits  the  message  to  all  destinations  on  the  net.  The 
source,  Dl  and  D2  drop  the  message  because  it  has  a  repeated  ID.  In  Figure  14,  we  assume  that 
D3  and  relay  R2  are  in  range  of  Rl  and  that  destination  D4  is  not.  In  JTIDS  radios,  sources  send 
on  their  time  slots  and  relays  are  assigned  an  equivalent  set  of  time  slots  that  immediately  follow 
the  source  time  slots.  Therefore,  the  first  hop  relay  transmits  in  the  time  slots  following  the 
transmission  of  ml  from  the  source.  Relay  R2  transmits  the  message  received  on  the  time  slots 
following  transmission  from  Rl;  at  the  same  time  the  source  can  transmit  the  next  message  m2. 

In  the  simulation,  destination  D3  will  be  scheduled  to  receive  ml  relayed  from  R2,  and  m2  from 
Rl .  In  the  figure  we  assume  that  R2  is  closer  to  D3  than  Rl .  Therefore  the  signal  from  R2  is 
stronger  than  that  from  Rl,  so  that  D3  considers  the  signal  from  R2  as  the  valid  signal  and  the 
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message  from  R1  as  “noise”.  Because  ml  has  already  been  received  from  Rl,  D3  will  drop  the 
message  from  R2.  This  is  an  example  where  a  single  and  double  relay  hop  to  a  single 
destination,  D3  in  this  case,  can  result  in  the  loss  of  a  valid  message. 


Relays  (multiple  relays  in  particular)  therefore  complicate  simulation  loading  in  several  ways: 

•  Relays  cause  additional  FPPS  calculations  and  less  predictable  message  transmissions. 

•  Multiple  relay  hops  to  a  destination  can  cause  loss  of  messages. 

•  Relays  will  not  retransmit  messages  they  have  already  transmitted;  therefore  it  is  less 
predictable  to  determine  the  number  of  relay  transmissions  to  expect  for  a  given  net. 

•  Because  many  relays  are  likely  to  be  fast  moving  airborne  platforms,  they  will  invoke  a 
higher  number  of  FPPS  calculations  than  ground-based  relays. 

JTIDS  increments  a  “Relay”  counter  each  time  that  a  relay  schedules  a  destination  to  receive. 
This  value  is  used  as  a  metric  for  measuring  additional  load  due  to  relays. 

3.2.4  HLA  Overhead  and  “Tick”  Time 

GSS  receives  HLA  events  via  the  HLA  “tick”  routine.  The  amount  of  time  spent  in  the  HLA  tick 
routine  is  specified  with  a  minimum  and  maximum  limit.  The  minimum  tick  time  is  incurred 
whenever  GSS  inspects  the  HLA  buffer  even  if  no  HLA  interactions  have  arrived.  If  interactions 
have  arrived,  then  GSS  will  pull  as  many  as  possible  off  the  HLA  buffer  within  the  maximum 
tick  interval.  Once  an  HLA  interaction  has  been  received  by  GSS,  it  is  automatically  processed  a 
user  specified  HLA  event  handler. 

ENTITYSTATE  messages  result  in  updates  to  platfonn  locations  in  the  platform  database,  and 
movement  of  platform  icons  on  the  display.  MSGSEND  interactions  start  a  sequence  of  tasks 
leading  up  to  a  potential  message  transmission.  First  the  Entity  ID  of  the  source  and  destination 
platforms  in  looked  up  and,  if  found,  a  search  is  done  to  find  a  Net  of  the  appropriate  type.  Once 


17 


a  net  is  found  then  its  Access  Mode  is  detennined  and  a  JTIDS  internal  message  is  constructed 
for  transmission  through  the  simulation.  This  message  includes  the  payload  required  for  the 
JSAF  message,  e.g.,  destination  ID,  JSAF  message  number  and  size. 

Early  experiments  indicated  that  the  minimum  HLA  tick  time  was  eating  a  large  portion  of 
available  real  time.  This  resulted  in  a  reduction  of  the  minimum  HLA  tick  time  that  is  detailed  in 
a  later  section.  Also,  experimentation  has  shown  that  HLA  message  handling,  once  received 
within  GSS,  does  not  introduce  a  substantial  load  on  the  JTIDS  simulation  compared  to  other 
factors  such  as  message  transmission.  This  is  described  in  more  detail  in  a  later  section. 
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3.3  Test  Configurations 


Figure  16  shows  the  main  test  configuration  that  was  used  for  experimentation.  The  GIESim 
JTIDS  under  test  ran  on  a  PC  with  no  other  applications  loaded  and  generated  a  Monitor  file  of 
data  collected  during  each  run.  A  specific  NET  file  was  loaded  based  on  the  type  of  test  being 
run.  For  the  majority  of  the  tests,  a  single  Dell  Desktop  PC  was  used  as  the  Driven  system.  The 
JTIDS  Driver  of  position  updates  (HLA  GIESIM  ENTITY  STATE  interactions)  also  ran  on  a 
separate  PC.  The  same  scenario  file  was  loaded  into  both  systems  for  each  load  run.  A  third  PC 
ran  three  applications,  an  HLA  Player,  an  HLA  Generator,  and  an  HLA  Recorder.  The  HLA 
Player  and  HLA  Generator  created  the  Link- 16  transmission  requests  (HLA 
GIESIM  MSG  SEND  interactions).  The  HLA  Recorder  logged  all  HLA  traffic  during  a  load 
run. 


Figure  17  shows  a  different  configuration  of  PCs  and  applications  that  were  used  to  record  test 
loads  for  later  playback  in  the  main  test  configuration.  Up  to  four  HLA  message  generators  were 
run  across  two  PCs  to  create  various  test  loads.  This  approach  simplified  load  generation,  since 
once  the  load  was  recorded  it  could  be  played  back  easily  and  consistently. 


19 


Figure  17  -  Set-up  Configuration  for  Initial  Script  Building 


Details  on  PCs  used  in  each  run  are  provided  in  Table  1  below.  As  mentioned  earlier,  the  Main 
Test  PC  was  used  for  the  majority  of  the  tests.  The  Dell  Laptop  (Alt  Test  1)  was  used  to  test  load 
on  a  very  weak  machine,  and  the  Custom  Desktop  PC  (Alt  Test  2)  was  used  to  measure 
performance  of  a  machine  with  more  memory  than  Main  Test  and  that  was  slightly  faster.  The 
Custom  Desktop  also  had  a  very  fast,  high-end  graphics  card,  and  two  runs  were  done  with 
GIESim  JTIDS  running  in  hardware  graphics  mode. 


Table  1  -  Computers  Used  in  Experimentation 


Purpose 

(PC  Type 

Processor 

OS 

RAM 

Video 

Main  Test 

Dell  Desktop 

P4  2.8  GHz 

Win  XP  SP2 

1  GB 

Integrated 

Rec/Gen 

Alt  Test  1 

Dell  Laptop 

P3  850 

MHz 

Win  2K  SP4 

256  MB 

ATI  Mobility  M4  32  MB  card 

Drive  PC 

Compaq  Laptop 

P4  2.8  GHz 

WinXPSPl 

896  MB 

ATI  Mobility  Radeon  9000  128  MB 

Alt  Test  2 

Custom  Desktop 

P4  3.0  GHz 

Win  2K  SP4 

2  GB 

ATI  Radeon  9800  Pro  256  MB  DDR 
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3.4  Run  Definitions 


This  section  provides  an  overview  of  the  scenarios,  networks  and  load  runs  that  were  designed 
and  executed  for  experimentation  with  JTIDS. 

3.4.1  Scenarios  for  Experimentation 

All  operational  scenarios  that  were  used  in  JTIDS  experimentation  are  based  on  derivatives  of 
the  PSI  Korona  Scenario  that  is  described  in  reference  [3].  Two  variations  of  Korona  were 
developed.  The  first  variation,  KORONADEMOREF.SCN,  is  the  same  as  the  basic  Korona 
scenario  with  the  addition  of  two  Global  Hawk  UAVs  that  were  used  for  reference.  The  second 
variation  of  Korona,  KORONA  HALF  REF.SCN,  contains  roughly  half  the  platforms  from  the 
basic  scenario  in  addition  to  the  two  reference  UAVs.  Table  2  summarizes  the  platform  force 
composition  in  both  test  scenarios. 


Table  2  -  Platform  Counts  for  two  Korona  Variations 


KORONA_DEMO_REF 


AB  IFC  Sensor 

2 

ABL 

4 

KC  135 

5 

Air  Cav  Helo 

2 

E2C 

6 

< / ) 

E 

E3 

4 

EP3 

1 

15 

F15F 

24 

CL 

F18 

24 

< 

JLENS 

3 

JSTARS 

2 

Rivet  Joint 

2 

Global  Hawk  UAV 

12 

Global  Hawk  Src/Dest  Refs 

2 

Total  Air 

93 

Carriers 

2 

ro 

Cruisers 

15 

C/3 

Submarine 

1 

Total  Sea 

18 

CAC2S 

6 

CRC 

2 

JTAGS 

4 

■O 

c 

Patriot  ICC 

4 

3 

o 

SHORAD 

2 

5 

TOAM 

2 

THAAD  TOC 

4 

UAV  Grnd  Station 

1 

Total  Ground 

25 

Moving 

108 

Stationary 

27 

Total  Link-16  Platforms 

1361 

KORONAHALFREF 


AB  IFC  Sensor 

2 

ABL 

2 

KC  135 

5 

Air  Cav  Helo 

2 

E2C 

6 

w 

E 

E3 

3 

o 

EP3 

1 

15 

F15F 

0 

CL 

FI  8 

0 

< 

JLENS 

0 

JSTARS 

1 

Rivet  Joint 

2 

Global  Hawk  UAV 

8 

Global  Hawk  Src/Dest  Refs 

2 

Total  Air 

34 

Carriers 

2 

ro 

Cruisers 

6 

CO 

Submarine 

1 

Total  Sea 

9 

CAC2S 

3 

CRC 

2 

JTAGS 

3 

■O 

c 

Patriot  ICC 

2 

3 

O 

SHORAD 

2 

5 

TOAM 

2 

THAAD  TOC 

2 

UAV  Grnd  Station 

1 

Total  Ground 

17 

Moving 

43 

Stationary 

17 

Total  Link-16  Platforms 

60  1 

Figure  88  shows  the  location  and  relative  positions  of  the  two  reference  platforms.  These 
platforms  were  added  to  support  a  Reference  Net  for  load  and  latency  analyses  during 
experimentation  runs.  The  full  complement  of  platforms  was  driven  with  position  updates 
during  load  testing  using  each  scenario.  The  next  section  covers  the  Reference  Net  and  other 
networks  that  were  used  in  experimentation. 
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3.4.2  Networks  for  Experimentation 


PSI  designed  a  set  of  operational  nets  for  the  Korona  scenario.  These  networks  were  designed 
based  on  high-level  suggestions  described  by  MITRE.  The  Korona  Networks  were  highly 
complex  and  contained  a  very  high  number  (12+)  of  relays  for  most  nets.  For  reasons  explained 
earlier,  large  numbers  of  relays  in  nets  introduce  extra  complexity.  For  this  reason,  and  to 
simply  experimentation  and  to  make  it  more  controllable,  several  sets  of  operational  nets  were 
designed  based  on  the  Korona  Net  design  with  either  no  relays  or  a  single  relay  each. 

We  started  the  net  design  process  by  stripping  out  all  relays  from  the  Korona  network.  Two 
main  net  groups  were  then  defined  as  shown  in  Table  3.  Nets  in  the  41  Net  Group  contain  all  the 
destinations  from  the  original  Korona  scenario.  We  used  the  Link- 16  Planning  Tool  to  find 
suitable  relays  for  the  20  nets  with  one  relay.  Relays  were  chosen  for  best  position  with  respect 
to  the  network  source  platforms.  Nets  in  the  8 1  Net  Group  were  based  on  the 
KORONAHALFREF.SCN  scenario  and  were  developed  by  loading  this  scenario  into  the 
Planning  Tool  and  then  loading  the  Korona  network  that  had  been  stripped  of  all  relays.  The 
Planning  Tool  automatically  dropped  missing  destinations  from  nets  and  dropped  nets  with 
missing  source  platforms.  The  resulting  network  was  stored  and  the  Planning  Tool  was  used  to 
add  in  suitable  relays  for  the  nets  that  we  wanted  to  have  a  relay. 
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Table  3  -  Net  Groups  for  Experimentation 


Net  Groups 

Nets  with  No  Relay 

Nets  with  One  Relay 

41  Net  Group 

KORONA  REF  RELAY.NET 
80-94  Destinations 

1  Ref  Net 

2  Groups  of  10: 

■  10  PPLI  B 

■  10  Mission  Management 

2  Groups  of  10: 

•  10PPLIB 
■  10  Mission  Management 

81  Net  Group 

KORONA  HALF  REF.NET 
48-56  Destinations 

1  Ref  Net 

2  Groups  of  20: 

■  20  PPLIB 

■  20  Mission  Management 

2  Groups  of  20: 

■  20  PPLI  B 

■  20  Mission  Management 

Appendix  A  provides  details  on  these  Net  Groups.  The  idea  behind  these  Net  groups  was  to 
allow  us  to  create  different  mixes  of  loads.  The  4 1  Net  group  and  8 1  Net  group  are  mutually 
exclusive  and  are  always  used  in  separate  runs  and  are  paired  with  their  respective  scenario  as 
shown  in  Table  4. 


Table  4  -  Scenario  and  Net  File  Pairing 


Scenario 

Net  File 

KORONA  DEMO  REF.SCN 

KORONA  REF  RELAY.NET 

KORONA  HALF  REF.SCN 

KORONA  HALF  REF.NET 

3.4.3  Load  Runs 

The  load  runs  were  to  some  extent  arrived  at  empirically  through  early  experimentation  that  was 
required  to  size  the  scope  of  final  experimentation  and  loads.  Many  early  runs  were  done  to 
shake  out  the  tools  and  to  determine  bounds  for  driving  the  simulation.  Many  refinements  and 
enhancements  to  JTIDS  and  the  associated  tools  were  identified  and  tested  in  this  early  phase. 
Once  we  had  collected  some  preliminary  data,  we  established  and  followed  a  plan  for 
experimentation. 

The  runs  fell  into  the  following  categories: 

•  Load  runs:  Used  a  single  target  PC  that  involved  stepped  loads  on  different  size  nets  and 
scenarios. 

•  Sensitivity  runs:  Explored  the  impact  on  platform  update  rate  on  load. 

•  PC  comparison  runs:  Perfonned  the  same  set  of  load  runs  on  three  different  PCs. 

•  Long-Term  Stability  run:  One  10-hour  run  was  done  with  a  heavy  load  to  determine 
the  stability  of  the  JTIDS  simulation. 

We  decided  to  use  different  load  rates  that  we  came  to  refer  to  as  “full”,  “half’  and  “quarter” 
rates  to  drive  the  nets.  For  each  “load”  rate,  we  used  different  values  for  nets  with  and  without 
relays.  Table  5  shows  that  network  load  rates  that  were  used  in  the  experimentation.  Generally, 
the  HLA  message  generator  was  set  to  generate  specified  messages  at  random  rates  between  the 
high  and  low  values  shown  in  the  table. 
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Table  5  -  Network  Load  Rates 


Nets  with  No  Relay 

Nets  with  One  Relay 

Full  Rate 

5-20  Seconds 

15-30  Seconds 

Half  Rate 

10-40  Seconds 

30  -  60  Seconds 

Quarter  Rate 

20  -  80  Seconds 

60-  120  Seconds 

For  most  load  runs,  we  decided  to  use  a  stepped  sequence  of  transmission  requests.  Every  five 
minutes  (real  time)  we  introduced  another  set  of  transmission  requests  on  a  new  net  group, 
building  up  to  a  total  of  four  net  groups  under  load.  Then  we  turned  off  net  group  requests  every 
five  minutes.  The  sequence  and  loads  for  41  Nets  and  for  81  Nets  is  shown  in  Table  6  and  Table 
7  respectively. 


Table  6-41  Net  Group  Stepped  Loads 


System  Load  -  Transmission  Requests 

11  Nets  -  No  Relays 

21  Nets  +  10  Relays 

31  Nets  +  10  Relays 

41  Nets  +  20  Relays 

31  Nets  +  10  Relays 

21  Nets  +  10  Relays 

11  Nets  -  No  Relays 

10  Nets  w / 1  Relay 


10  Nets  -  no  Relays 


10  Nets  w/ 1  Relay 


10  Nets  -  no  Relays  +  Ref  Net 


Table  7-81  Net  Group  Stepped  Loads 


System  Load  -  Transmission  Requests 

21  Nets  -  No  Relays 

41  Nets  +  10  Relays 

61  Nets  +  10  Relays 

81  Nets  +  20  Relays 

61  Nets +  10  Relays 

41  Nets  +  10  Relays 

21  Nets  -  No  Relays 

20  Nets  w/ 1  Relay 


20  Nets  -  no  Relays 


20  Nets  w/ 1  Relay 


20  Nets  -  no  Relays  +  Ref  Net 


5  Min  |  10  Min  |  15  Min  |  20  Min  |  25  Min  |  30  Min  |  35  Min 


Note  that  these  loads  were  not  uniform  in  size  due  to  the  fact  that  every  other  net  group  that  was 
introduced  had  nets  with  a  single  relay.  Also,  the  nets  within  the  Net  Groups  varied  in  terms  of 
their  throughput  with  the  Mission  Management  nets  having  an  RRN  of  8  and  the  PPLI  B  nets 
having  an  RRN  of  7.  Note  that  the  Reference  Net  has  an  RRN  of  8. 

The  stepped  requests  allowed  us  to  predict  the  ideal  shape  of  the  cumulative  count  of 
MSG  SEND  interactions  (transmission  requests)  as  shown  in  the  pink  curve  in  Figure  .  If  the 
JTIDS  system  is  keeping  up  with  real  time,  then  the  shape  of  the  cumulative  count  of 
MSG  RCVD  interactions  (transmission  responses)  should  have  exactly  the  same  shape.  The 
lower  black  curve  shows  the  stepped  request  rates  during  each  five-minute  step  period. 
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Cumulative  Countvs  Stepped  Drive  Rate  of  (1 ,2, 3, 4, 5, 4, 3,2,1 ,0) 


Figure  19  -  Ideal  Cumulative  Count  Curve  for  Stepped  Transmission  Requests. 


Figure  20,  Figure  21  and  Figure  22  show  sample  transmission  request  rates  for  “full”,  “half’  and 
“quarter”  rates  respectively.  The  left  side  of  each  figure  shows  data  for  the  rate  with  no  relay, 
and  the  right  side  shows  data  for  the  rate  with  one  relay.  This  data  comes  from  the  LOG  files 
created  by  the  HLA  Recorder  from  recording  different  experimentation  runs. 
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Figure  20  -  Sample  of  Full  Request  Rates 


Figure  21  -  Sample  of  Half  Request  Rates 
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3.5  Metrics 


Metric  data  for  GIESim-JTIDS  bench-test  load  runs  were  collected  in  the  JTIDS  Monitor  MON 
files  and  in  the  GIESim  HLA  Recorder  LOG  files.  Some  data  in  these  files  was  used  directly, 
and  some  of  the  data  was  post  processes.  Examples  of  data  that  is  used  directly  include 
cumulative  counts  of  transmission  requests  in  the  Monitor  file  and  measured  latency  recorded  in 
the  recorder  LOG  file.  Some  post  analysis  involved  statistical  analysis  of  captured  data  to  yield 
new  metrics  for  understanding  JTIDS  loading  and  to  verify  correct  operation. 


Table  8  -  Notations  and  Meaning 


Notation 

Meaning 

<X> 

Average  value  of  a  measured  metric  over  the  duration  of  a  load  run.  Typically  applied  to 
measured  latency  values. 

SD(X) 

Standard  deviation  of  a  measured  metric  over  the  duration  of  a  load  run.  Typically  applied  to 
measured  latency  values. 

TMA(X) 

Time  moving  average  of  a  measured  metric  over  a  moving  window  of  time  -  typically  60  seconds 
unless  otherwise  specified.  TMA  is  typically  applied  to  cumulative  metric  data  to  determine  the 
average  driving  rate  over  the  averaging  window. 

Min(X) 

Minimum  value  of  a  measured  metric  over  the  span  of  a  load  run. 

Max(X) 

Maximum  value  of  a  measured  metric  over  the  span  of  a  load  run. 

Table  9  -  Metric  Notation  for  Data  Collected  by  JTIDS  Monitor 


Metric 

Notation 

Meaning 

How  Collected 

E 

Expected  value  -  JTIDS  internal  cumulative  measure  of  how 
many  receivers  are  expected  to  receive  messages  as  a  result  of 
transmissions. 

JTIDS/Monitor 
Accumulated  each  second  as 
transmission  requests  come  in. 

R 

Relay  value  -  JTIDS  internal  cumulative  measure  of  how 
many  relay  transmission  occur. 

JTIDS/Monitor 
Accumulated  each  second  as 
relay  transmissions  occur. 

E+R 

Sum  of  Expected  and  Relay  values. 

JTIDS/Monitor 
Accumulated  each  second. 

HR 

Cumulative  value  of  Host  messages  received. 

JTIDS/Monitor 
Accumulated  each  second  for 
host  receptions. 

S/R 

Ratio  of  simulation  clock  to  wall  (system)  clock. 

JTIDS/Monitor 

Computed  each  real  second. 

OF 

Cumulative  count  of  transmit  buffer  overflows  that  may 
occur. 

JTIDS/Monitor 
Accumulated  each  second  if 
overflow  occurs. 

ES 

Cumulative  count  of  Entity  State  HLA  interactions  received  to 
update  platform  positions. 

JTIDS/Monitor 

Computed  each  real  second. 

MS 

Cumulative  count  of  GIESIM  MSG  SEND  HLA  interactions 
that  request  message  transmissions. 

JTIDS/Monitor 

Computed  each  real  second. 

MR 

Cumulative  count  of  GIESIM  MSG  RCVD  HLA  interactions 
sent  by  JTIDS  when  a  requested  destination  has  received  the 
message. 

JTIDS/Monitor 

Computed  each  real  second. 

MS-MR 

Difference  in  total  requests  versus  responses  (lost  messages). 

Computed  in  Monitor. 

T 

Total  duration  of  the  run  in  seconds. 

JTIDS/Monitor. 
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The  following  table  lists  post  process  analysis  of  JTIDS  Monitor  data. 


Table  10  -  Metrics  Computed  from  Monitor  Data 


Computed 

Metric 

Notation 

Meaning 

<E> 

Average  value  of  E  over  the  duration  of  a  load  run. 

TMA(E) 

Time  moving  average  of  Expected  values. 

<R> 

Average  value  of  R  over  the  duration  of  a  load  run. 

TMA(R) 

Time  moving  average  of  Relay  values. 

<E+R> 

Average  value  of  E+R  over  load  run. 

TMA(E+R) 

Time  moving  average  of  E+R. 

The  GIESim  HLA  Recorder  captures  all  GIESirn  HLA  interactions  that  occur  and  time  stamps 
them  with  the  wall  clock  (system  time)  in  which  they  occur.  The  Recorder  can  record  all  of  the 
five  GIESim  HLA  interactions  and  their  data  fields  that  have  been  defined.  Several  post  analysis 
operations  are  performed  on  recorded  data  to  yield  additional  metrics.  These  are  shown  in  the 
table  below. 


Table  11  -  Metrics  Computed  from  Recorder  LOG  Files 


Computed 

Metric 

Notation 

Meaning 

<Net> 

Latency  values  reported  for  selected  nets  average  over  the  load  run. 

SD(Net) 

Standard  deviation  of  reported  latencies  for  a  selected  net. 

Min(Net) 

Minimum  latency  for  a  selected  net. 

Max(Net) 

Maximum  latency  for  a  selected  net. 
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3.6  Charts 


With  the  large  number  of  metrics  that  were  collected,  there  are  an  equally  large  number  of 
possible  charts.  Charts  are  useful  in  understanding  the  dynamics  of  loading  and  the  state  of  the 
simulation  at  different  points  in  time,  and  the  externally  observable  results  recorded  in  the  HLA 
recorder.  Charts  (and  analyses)  tended  to  fall  into  two  categories  as  shown  below. 

•  Run  Charts:  We  decided  to  “standardize”  on  the  following  charts  for  each  run  with  data 
spanning  the  duration  of  the  run: 

o  Sim/Real  Ratio  (MON  data) 
o  Ref  Net  Latency  (LOG  data) 

o  Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  interactions  (MON  data) 
o  Cumulative  Expected  (E)  and  Relays  (R)  (MON  data) 
o  Time  Moving  Averages:  TMA(MS)  and  TMA(MR)  (MON  data) 
o  Time  Moving  Averages:  TMA(E)  and  TMA(R)  (MON  data) 

We  occasionally  selected  other  metrics  to  chart  for  comparison  and  analysis,  e.g.,  latency 
of  other  nets. 

•  Cross-run  Charts:  These  were  used  to  compare  and  analyze  load  factors  between  runs. 
Some  examples  include: 

o  Cross  Run  Comparison  Spreadsheet  of  collected  metric  data  on  the  target  PC,  for 
different  position  update  rates,  and  between  target  test  PCs. 
o  Sensitivity  Charts:  Plot  a  measure  of  load  against  Sim/Real  Ratio. 

•  Special  Case  Analysis  Charts:  These  are  charts  that  were  used  to  study  specific  things 
such  as: 

o  HLA  Overhead, 
o  Position  update  rates  and  modes, 
o  Transmission  Request  rates  for  selected  nets. 

Early  charting  and  comparison  of  Monitor  and  Recorder  data  gave  us  confidence  that  the 
Monitor  and  Recorder  were  working  correctly  and  that  internal  and  external  measures  correlated 
very  well. 
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4.0  JTIDS  Enhancements 


4.1  Instrumentation  and  Monitoring 

The  PSI  JTIDS  simulation  was  designed  and  built  with  performance  measurement  capabilities, 
which  were  “inherited”  by  the  GIESim  version  of  JTIDS.  To  support  the  experimentation 
associated  with  loading  of  the  GIESim  JTIDS  simulation,  some  of  these  functions  in  addition  to 
other  metrics  associated  with  HLA  were  instrumented  and  monitored.  This  effort  required  a 
small  amount  of  effort  to  achieve.  Some  extra  statements  were  added  to  key  places  in  the  JTIDS 
simulation  and  to  keep  performance  data.  This  data  structure  is  tracked  by  a  new  GSS-based 
monitor  task  that  is  started  by  the  JTIDS  simulation  and  runs  in  parallel  with  it.  Figure  23 
illustrates  the  JTIDS  instrumentation  and  monitor.  JTIDS  launches  the  monitor  task 
automatically  whenever  it  starts.  The  monitor  task  schedules  a  process  to  periodically  read  the 
metrics  data  being  collected  in  JTIDS.  The  Monitor  task  does  minimal  processing  on  the  data 
until  JTIDS  ends,  at  which  point  the  Monitor  writes  its  stored  data  to  a  MON  file.  A  Sim  Clock, 
which  also  displays  the  current  Sim/Real  Ratio,  was  also  added  to  JTIDS.  This  clock  allowed  us 
to  determine  how  well  JTIDS  was  maintaining  real  time. 

The  monitor  task  is  simple,  small  and  was  easy  to  build.  The  Monitor  task  architecture  is  shown 
in  Figure  104.  Instrumentation  and  monitoring  of  JTIDS  were  keys  to  the  experimentation  that 
was  performed.  Load  testing  indicated  that  the  instrumentation  and  Monitor  functions  introduce 
a  negligible  load  on  the  system  PC. 


Figure  23  -Instrumentation  and  Monitoring  of  JTIDS  for  Load  Test  Experimentation 


Figure  24  indicates  the  metrics  that  are  accumulated  in  JTIDS  that  are  collected  by  the  Monitor. 
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Figure  24  -  New  Monitor  Task  for  Collecting  JTIDS  data 

The  MON  file  is  automatically  assigned  a  unique  name  of  the  form: 
MON_050714_13584804.CSV.  Monitor  files  store  data  in  comma-separated-value  (CSV) 
format  that  can  be  directly  read  into  Excel. 

The  next  section  provides  additional  infonnation  on  enhancements  and  updates  that  were 
accomplished  for  the  GIESim  JTIDS  Simulation. 
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4.2  Enhancements  to  GIESim  JTIDS 


During  early  experiments  with  GIESim  JTIDS,  several  needed  enhancements  were  identified  that 
could  substantially  increase  load  capacity  and  accuracy  of  responses.  These  enhancements  were 
beyond  the  instrumentation  that  was  added  to  JTIDS.  Enhancements  to  JTIDS  are  summarized 
in  the  list  that  follows. 

•  GIESim  Enhancements:  Figure  25  shows  the  architecture  of  the  updated  and  refined 
GIESIM  INTERFACES  model  that  now  includes  the  new  INSTRUMENTMODEL. 
Several  enhancements  were  made  to  this  interface  module  that  improved  performance.  In 
particular,  the  efficiency  of  the  Entity  ID  search  processes  was  improved  and  the  net 
search  efficiency  was  dramatically  improved  by  using  previously  computed  information. 

•  Instrumentation:  Details  of  instrumentation  were  largely  discussed  in  the  prior  section. 
Figure  126  provides  a  close-up  of  the  INSTRUMENT_MODEL  architecture. 

•  Reduced  HE  A  tick  time:  The  minimum  time  to  process  HLA  events  within  the  HLA 
tick  routine  is  referred  to  as  the  “tick”  time.  GSS  had  a  minimum  tick  default  value  of  1 
ms.,  which  had  been  set  for  other  GSS  applications  of  HLA.  Due  to  the  large  amount  of 
processing  performed  in  the  JTIDS  simulation,  the  minimum  tick  time  was  being  invoked 
frequently  and  taking  up  a  lot  of  real  time.  We  assessed  the  impact  of  reducing  the 
minimum  tick  interval  and  eventually  chose  100  us.  This  value  had  no  impact  on  HLA 
event  handling  and  made  a  huge  difference  in  available  real  time. 

•  Reduced  overhead  of  HLA  GUI:  The  GIESim  JTIDS  simulation  has  a  GUI  to  show 
HLA  traffic  and  cumulative  counts  of  each  GIESim  HLA  interaction.  This  panel  was 
updated  (redisplayed)  for  each  HLA  event,  either  sending  or  receiving.  This  display 
activity  introduced  a  high  load  on  the  system,  which  diminished  the  ability  of  JTIDS  to 
handle  message  requests.  This  panel  is  now  updated  every  5  seconds,  which  significantly 
improves  available  real-time. 

•  New  Latency  Calculation:  The  latency  calculation  was  changed  to  use  wall  (system) 
clock  rather  than  simulation  clock  since  we  were  interested  in  the  real  rather  than 
simulated  latency.  In  early  applications  of  GIESim  JTIDS  with  JSAF,  only  single 
transmission  requests  were  being  made,  and  JTIDS  was  running  essentially  at  real  time. 
However,  as  JTIDS  gets  loaded  down  the  simulation  clock  can  fall  behind  the  wall  clock. 
The  use  of  wall  clock  time  to  determine  message  latency  provides  a  more  accurate 
measure  of  the  real  performance  of  JTIDS  under  load. 

•  Migration  to  JTIDS  1.4Update:  The  GIESim  JTIDS  Simulation  was  migrated  to  the 
most  recent  version  of  the  PSI  JTIDS  Simulation  (1.4Update).  This  move,  while  costly 
in  time,  was  deemed  to  take  less  time  than  to  find  and  fix  bugs  that  were  already  fixed  in 
the  latest  JTIDS.  This  move  fixed  a  number  of  problems  and  provided  new  capabilities: 
o  The  link  states  to/from  ground  platforms  now  accurately  reflect  their  true  states.  The 

older  version  of  JTIDS  that  GIESim-JSAF  was  based  on  sometimes  did  not  update 
link  states  for  ground  platforms,  which  could  cause  messages  to  drop. 
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o 


o 


The  newer  JTIDS  simulation  has  a  new  NET  file  format,  therefore  by  moving 
GIESim-JSAF  to  the  newer  version  we  can  now  use  all  the  capabilities  of  the  current 
Link- 16  NMS  including: 

■  Link- 16  Planning  tool 

■  Time  Slot  Allocation  tool. 

Migration  to  the  newer  JTIDS  and  GSS  improves  support  of  GIESim  JTIDS. 


GIESin_ INTERFRCES 


COuEC’CP 

45 

Figure  25  -  Updated  and  Enhancement  GIESIMINTERFACES  Model  in  JTIDS 


I NSTRUMENT_ MODEL 


Figure  26  -  New  Instrumentation  Model  added  to  JTIDS 
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5.0  Driver  Enhancements 


During  early  experimentation,  we  detennined  that  more  controlled  position  updates  were 
required  from  the  JTIDS  Driver  simulation.  We  also  identified  some  modes  of  update  operations 
that  would  be  good  to  have  for  load  testing,  and  some  opportunities  to  reduce  load  on  the  Driver 
system.  As  a  consequence  the  Driver  was  updated  as  listed  below. 

■  New  Platform  Update  Model:  A  new  platform  update  model  was  introduced  that  had  been 
built  for  another  application.  This  update  model  provides  very  precise  control  of  update 
rates.  In  addition,  the  update  model  was  enhanced  to  provide  new  modes  of  operation. 

Mode  of  operation  and  update  rate  are  now  specified  in  the  file  PUVARS.SFI.  The  new 
modes  of  operation  are: 

•  Update  Platforms: 

o  All  platfonn  locations  -  sends  updates  for  all  platforms, 
o  Changed  platform  locations  -  sends  updates  only  for  platfonns  that  change 
location.  The  initial  update  will  send  all  platform  positions. 

•  Update  Type: 

o  Bulk  -  All  position  updates  are  sent  “at  once”  at  each  update  interval.  In  actuality 
it  may  take  2-3  seconds  to  send  130+  updates, 
o  Distributed:  Position  updates  are  spread  uniformly  over  the  update  interval. 

■  Reduced  HU  A  GUI  Overhead:  The  HLA  GUI  is  now  updated  after  completion  of  the 
platform  update  cycle  rather  for  every  HLA  event. 

■  Reduced  1 1 1  A  Tick:  The  HLA  minimum  time  was  reduced  to  100  us. 

■  Update  Only  Link-16  Platforms:  The  Link-16  NMS  scenario  file  can  contain  logical 
references  to  hierarchical  platform  groups  such  as  a  mission.  These  logical  references  are 
represented  as  icons  on  the  JTIDS  screen.  While  sending  position  updates  for  these 
references  does  not  cause  a  problem,  it  did  add  to  the  HLA  message  overhead  so  the  Driver 
was  modified  to  only  send  position  updates  for  actual  Link- 16  platforms. 

■  GSS  Upgrade:  The  Driver  is  now  built  with  GSS  Release  10.3. 10. 

Sample  runs  using  all  four  Driver  update  combinations  were  performed,  and  the  cumulated 
ENTITY  STATE  counts  were  analyzed  and  charted  from  the  JTIDS  Monitor  data  files  as  shown  in 
Figure  137  through  Figure  30. 

In  private  discussions  with  Jerry  Reaper  at  SAIC,  it  was  detennined  that  JSAF  processes  and 
sends  position  updates  for  each  platfonn  separately.  However,  platform  position  updates  occur 
on  a  fairly  regular  schedule  such  that  all  position  updates  are  typically  sent  very  close  together 
every  10  seconds.  Furthermore,  JSAF  sends  position  updates  for  all  platforms  even  if  the 
platforms  have  not  changed  position.  Therefore,  for  the  majority  of  our  experiments  we  chose  to 
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set  the  Driver  simulation  to  send  ALL  positions  every  10  seconds  as  shown  in  Figure  27.  In 
selected  runs  we  examined  the  impact  of  using  Changed  and  Distributed  updates. 


Cum  Entity  States  recorded  in  Monitor 
(Driver  sending  ALL  positions  in  Bulk  updates  every  10  sec.) 


Figure  27  -  Cumulative  Driver  Updates:  ALL  positions  every  10  seconds 


Figure  28  -  Cumulative  Driver  Updates:  ALL  updates  distributed  over  10  seconds. 
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Figure  29  -  Cumulative  Driver  Updates:  Changed  positions  every  10  seconds 


Figure  30  -  Cumulative  Driver  Updates:  Changed  positions  distributed  over  10  seconds 


36 


6.0  HLA  Recorder/Player/Generator  (HRPG)  Tool 

Based  on  the  experience  in  preparation  for  the  Mar  05  SPIE  demonstration  of  GIESim/JSAF 
integration,  PSI  and  SAIC  identified  the  need  for  a  GIESim  HLA  Recorder/Player  (HRP).  This 
capability  would  allow  SAIC  to  record  JSAF  messages  being  sent  to  JTIDS  and  responses  sent 
from  JTIDS.  Recordings  would  then  be  sent  to  PSI  for  playing  into  JTIDS  for  the  purposes  of 
debugging,  and  more  importantly  for  interoperability  load  experiments.  An  initial  version  of 
HRP  was  developed  by  PSI  and  sent  to  SAIC  on  May  12,  2005.  The  HRP  is  based  on  the  HLA 
test  tool  that  PSI  developed  for  prior  GIESim  projects.  The  recorder  and  player  functionally  of 
HRP  were  adapted  from  HLA  work  that  PSI  did  for  the  Army. 


Create/Modify  Interaction 


Generate  Mode  Can 
Run  Concurrently  With 
the  Play  Modes 


HLA 


ec/Player/Gen 


HRPG 


GIEsim 

HLA 

Rec/Player 

Generator 

(HRPG) 


Figure  31  -Functionality  and  Graphics  of  the  HLA  Recorder/Player/Generator  (HRPG) 


During  initial  planning  for  experimentation  on  JTIDS,  PSI  realized  that  HFP  could  be  extended 
to  include  the  ability  to  generate  messages.  This  is  a  straightforward  extension  of  HRP  and 
resulted  in  a  new  tool  -  HRPG.  The  Generate  mode  of  HRPG  can  be  used  concurrently  with  the 
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Play  mode.  Figure  3 1  illustrates  the  operating  modes  and  screens  of  FIRPG.  The  upper  part  of 
the  Figure  3 1  illustrates  the  Generate  mode  function.  FIRPG  allows  creation  and  scheduling  of 
any  of  the  GIESim  HLA  interactions  on  either  a  one-shot,  periodic  or  quasi-random  rate. 
Multiple  messages  with  different  content  can  be  scheduled.  FIRPG  allows  modification  or 
deletion  of  any  running  message  generator.  HRPG  has  proven  to  be  an  essential,  invaluable  tool 
for  bench-test  experimentation  on  JTIDS. 

The  Figure  32  is  a  screen  shot  of  the  architecture  of  HRPG  that  shows  its  functional  portioning. 


Figure  32  -  GSS  Architectural  Drawing  of  the  Final  FILA  Recorder/Player/Generator 
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The  following  list  provides  an  overview  of  HRPG  functions: 

•  HLA  Message  Generator:  Functions  allow  a  user  to  build  messages  for  each  GIESim 
HLA  interaction.  Up  to  50  user  created  interactions  of  each  type  can  be  stored. 

o  Transmission  options  include: 

■  Single  message. 

■  Periodic  transmissions 

■  Pseudo  random  transmissions. 

o  Auto  ID  numbering:  In  any  message,  if  the  message  ID  is  set  to  zero,  then 
HRPG  automatically  supplies  a  sequential  message  ID  from  a  sequence  that  it 
maintains.  Auto  ID  capability  is  extremely  useful  in  tracking  specific  messages 
and  associated  responses  from  JTIDS. 

o  Modify/Cancel  Messages:  HRPG  creates  a  list  of  user  created  messages  and 
allows  the  user  to  select  and  modify  (or  delete)  the  message. 

•  Play  Mode:  Presents  the  user  with  a  list  of  recordings  to  play  and  plays  the  recording 
that  is  selected.  Prior  to  and  during  playing,  the  user  can  select  which  of  the  HLA 
interactions  to  play.  This  allows  playback  of  just  the  MSGSEND  interactions  to 
stimulate  the  JTIDS  simulation. 

•  Record  Mode:  This  mode  records  all  GIESim  HLA  interactions  that  occur  and 
automatically  stores  the  results  in  recording  file  with  a  name  of  the  form 

LOG  050717  15001301  into  the  LOG  FILE  sub  directory.  The  name  of  each  recording 
file  is  unique  since  it  includes  the  date  and  time  at  the  start  of  the  recording.  LOG  files 
record  data  in  comma-separated-value  (CSV)  fonnat  that  can  be  directly  read  into  Excel. 

Additional  enhancements  that  were  made  to  HRGP  over  the  period  of  experimentation  include: 

•  Reduced  HLA  Tick  time  to  100  us. 

•  Reduced  overhead  of  HLA  GUI  by  refreshing  it  every  5  seconds. 

•  HRPG  is  now  built  with  GSS  Release  10.3. 10. 
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7.0  Experimentation 


Over  26  hours  of  testing  was  required  to  run  the  22  load  tests  that  were  performed  to  collect  well 
over  100  MB  of  data  that  was  used  in  this  report.  Data  reduction,  charting  and  cross-run 
analyses  incurred  considerable  additional  time.  This  does  not  count  the  dozen  or  so  early  runs 
and  hours  of  testing  that  were  done  to  refine  the  tools  and  to  establish  the  overall  test 
methodology. 

Table  12  presents  all  the  load  runs  that  were  perfonned  on  the  Main  test  PC  and  that  summarizes 
the  high-level  data  that  was  collected. 


Table  12  -  Load  Runs  on  Main  Test  PC 


GIESim-JTIDS  Bench  Test  Data 

Load  Tests  on  Main  Test  PC 


Load  Run  Description 

Ref  Net 

10s  Rate 

41  Net 
Step 
.25  Rate 

41  Net 
Step 
.5  Rate 

41  Net 
Step 
Full  Rate 

41  Net 

Constant 

.5  Rate 

41  Net 
Step 
Full  Fate 

81  Net 
Step 
.5  Rate 

81  Net 
Step 

Full  Rate 

Duration  (sec) 

712 

3510 

2250 

2516 

2237 

2586 

2312 

2525 

Position  Updates 

Update  Rate  (sec) 

10 

10 

10 

10 

10 

10 

10 

10 

ALL/Delta 

ALL 

ALL 

ALL 

ALL 

ALL 

Delta 

ALL 

ALL 

Burst/Distrib 

Bulk 

Bulk 

Bulk 

Bulk 

Bulk 

Bulk 

Bulk 

Bulk 

Load  Factors 

l 

JTIDS  Simulation 

Internal  Measures 

Total  Expected 

61 

115752 

142429 

285034 

199489 

285034 

167371 

346726 

Total  Relays 

0 

27898 

34836 

67591 

67810 

67412 

39912 

78299 

Total  E+R 

61 

143650 

177265 

352625 

267299 

352446 

207283 

425025 

<Expected> 

0.086 

33 

63 

113 

89.18 

110 

72.4 

137.3112 

<Relays> 

0 

8 

15 

27 

30.31 

26 

17 

31.00815 

<E+R> 

0.086 

41 

79 

140 

119.49 

136 

89.7 

168.3194 

Total  Receives 

61 

79311 

96511 

191763 

136390 

191665 

79761 

163572 

<Receives> 

0.086 

23 

43 

76 

60.97 

74 

34.5 

64.77817 

Measured  Values 

recorded  from 

Monitor 

Total  ES 

9472 

56376 

35478 

40014 

31968 

28529 

17325 

18942 

<ES> 

13.3 

16 

16 

16 

14.3 

11 

7.49 

7.50 

Total  MS 

61 

1628 

1806 

3400 

2400 

3400 

3376 

6755 

<MS> 

0.086 

0.46 

0.80 

1.35 

1.07 

1.31 

1.46 

2.68 

Total  MR 

61 

1599 

1781 

3361 

2351 

3357 

3300 

6618 

<MR> 

0.086 

0.46 

0.79 

1.34 

1.05 

1.30 

1.43 

2.62 

Gap(MS-MR) 

0 

29 

25 

39 

49 

43 

76 

137 

1 

%  Lost 

0% 

2% 

1% 

1% 

2% 

1% 

2% 

2% 

Worst  SIR  Ratio 

1 

0.996 

0.883 

0.531 

0.576 

0.541 

0.993 

0.63 

Measures 

1 

<Ref  Net  Latency>| 

1.56 

1.63 

2.13 

5.28 

3.88 

4.76 

1.82 

2.85 

II  STDV(Ref  Net  Latency)! 

0.96 

0.89 

1.66 

5.45 

2.72 

4.65 

0.88 

2.37 

Min(Ref  Net  Lat) 

0.42 

0.23 

0.04 

0.11 

0.05 

0.05 

0.22 

0.04 

Max(Ref  Net  Lat) 

3.48 

4.60 

10.93 

26.20 

13.50 

27.77 

4.10 

12.79 
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Table  13  lists  load  runs  that  were  performed  to  examine  load  sensitivity  to  position  update  rates. 
Data  from  some  runs  from  Table  12  were  repeated  in  this  table  so  that  they  can  be  more  easily 
compared  on  a  side-to-side  basis. 


Table  13  -  Position  Update  Sensitivity  Runs 


GIESim-JTIDS  Bench  Test  Data 

Sensitivity  to  Position  Update  Rates 


5  Update  Rates 

3  Update  Rates  ) 

Load  Run  Description 

41  Net 
Step 
.5  Rate 

18s 

41  Net 
Step 
.5  Rate 

14s 

41  Net 
Step 
.5  Rate 

10s 

41  Net 
Step 
.5  Rate 

6s 

41  Net 
Step 
.5  Rate 

2s 

81  Net 
Step 
Full  Rate 

14s 

81  Net 
Step 
Full  Rate 

10s 

81  Net 
Step 
Full  Rate 

6s 

Duration  (sec) 

2260 

2240 

2274 

2306 

2272 

2443 

2525.11 

3031 

Position  Updates 

Update  Rate  (sec) 

18 

14 

10 

6 

2 

14 

10 

6 

ALL/Delta 

ALL 

ALL 

ALL 

ALL 

ALL 

ALL 

ALL 

ALL 

Burst/Distrib 

Bulk 

Bulk 

Bulk 

Bulk 

Bulk 

Bulk 

Bulk 

Bulk 

Load  Factors 

JTIDS  Simulation 

Internal  Measures 

Total  Expected 

142429 

142429 

142429 

142429 

142429 

346726 

346726 

346726 

Total  Relays 

34922 

34836 

34836 

34836 

34664 

77907 

78299 

78579 

Total  E+R 

177351 

177265 

177265 

177265 

177093 

424633 

425025 

425305 

<Expected> 

63.03 

63.58 

62.6 

61.76 

62.7 

142 

137.3112 

114.4 

<Relays> 

15.46 

15.55 

15.32 

15.1 

15.26 

31.9 

31.00815 

26 

<E+R> 

78.49 

79.1 

77.95 

76.87 

77.96 

173.8 

168.3194 

140.3 

Total  Receives 

96647 

96438 

96437 

96394 

95876 

163087 

163572 

163628 

<Receives> 

43.05 

42.4 

41.8 

42.2 

66.75 

64.77817 

54 

Measured  Values 

recorded  from 

Monitor 

Total  ES 

18056 

23088 

32708 

55352 

163688 

13090 

18942 

37884 

<ES> 

7.99 

10.31 

14.38 

24 

72.06 

5.36 

7.501455 

12.5 

Total  MS 

1806 

1806 

1806 

1806 

1806 

6755 

6755 

6755 

<MS> 

0.8 

0.806 

0.794 

0.783 

0.8 

2.765 

2.675131 

2.23 

Total  MR 

1782 

1781 

1780 

1780 

1778 

6614 

6618 

6605 

<MR> 

0.788 

0.795 

0.783 

0.772 

0.78 

2.71 

4.844802 

2.18 

Gap(MS-MR) 

24 

25 

26 

26 

28 

141 

137 

150 

1 

%  Lost 

1.3% 

1.4% 

1.4% 

1.4% 

1 .6% 

2.1% 

2.0% 

2.2% 

Worst  SIR  Ratio 

0.988 

0.98 

0.89 

0.84 

0.773 

0.753 

0.63 

0.553 

Measures 

1  1 

<Ref  Net  Latency>| 

1.77 

1.81 

1.87 

2.44 

3.10 

2.42 

2.85 

3.63 

||  STDV(Ref  Net  Latency)! 

1.18 

1.18 

1.29 

2.03 

3.40 

1.91 

2.37 

3.08 

Min(Ref  Net  Lat) 

0.04 

0.04 

0.05 

0.25 

0.05 

0.09 

0.04 

0.11 

Max(Ref  Net  Lat) 

7.22 

7.56 

7.20 

12.70 

17.02 

9.37 

12.79 

18.83 
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Table  14  lists  high-level  data  for  cross  PC  comparison  load  runs  plus  data  for  the  10-hour  long¬ 
term  stability  run. 


Table  14  -  Cross  PC  Comparison  Load  Runs  and  Long-Term  Stability  Run 

GIESim-JTIDS  Bench  Test  Data 


Cross-PC  Load  Test  Comparisons  &  Long-Term  Stability  Run 


Stopped 
HWG  No 
Affect 


The  analysis  of  data  collected  from  these  runs  is  presented  in  the  next  section.  Suggestions  for 
future  work  opportunities  are  presented  in  the  section  that  follows.  Overall  conclusions  from 
load  test  experimentation  are  presented  in  the  remaining  section.  The  “standard”  charts  for  the 
experimentation  runs  listed  in  the  tables  above  are  included  as  Appendix  B.  Note  that  all  data 
collected  is  being  provided  on  a  CD  that  is  hierarchy  organized  to  match  the  organization  of  the 
three  tables  above. 
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8.0  Analysis  of  Results 


PSI  collected  an  enormous  volume  of  data  from  the  runs  that  were  perfonned.  This  section 
provides  high-level  analyses  of  the  load  runs.  First,  an  analysis  is  performed  to  examine  the 
impact  of  HLA  and  system  overhead  to  latencies.  Next,  data  and  analyses  of  the  load  runs 
performed  on  the  Main  Test  PC  are  presented.  An  analysis  across  the  main  load  runs  follows 
next.  Finally,  results  from  position  update  rate  sensitivity  runs  are  presented,  followed  by 
analysis  of  the  cross-PC  load  tests. 

8.1  HLA  &  System  Overhead 

There  was  a  question  and  concern  about  the  possible  overhead  of  HLA  and  JTIDS  system 
message  handling.  Figure  33  illustrates  some  of  the  JTIDS  simulation  components  that  are 
involved  with  message  handling  and  transmission  latency.  The  figure  also  shows  how  data  was 
collected  to  explore  the  answer  to  the  question  of  overhead. 


Figure  33  -  HLA  and  System  Overhead  on  Message  Latency 


The  HLA  recorder  records  the  time  of  each  MSG_SEND  interaction  and  the  time  of  arrival  of 
each  associated  MSG_RCVD  interaction.  The  difference  in  arrival  times  provides  an  overall 
measure  of  total  system  latency  including  contributions  from  HLA  RTI-S.  The  latency  for  each 
message  “transaction”  was  then  subtracted  from  the  computed  time  difference  to  provide  a  result 
that  represents  the  contribution  of  the  JTIDS  and  HLA  without  the  internal  JTIDS  transmission 
latency. 

Figure  34  shows  the  HLA  and  system  overhead  for  the  Reference  Net  when  JTIDS  is  minimally 
loaded;  the  only  transmission  requests  are  the  Reference  Net  while  all  136  platforms  are  being 
updated.  The  overhead  variation  is  only  2  seconds  with  an  average  close  to  1.5  seconds. 

Figure  35  shows  a  similar  chart.  In  this  case  the  system  load  is  half  rate  on  the  41  Net  step  case 
while  all  platforms  are  updating  at  a  10  second  rate.  The  overhead  variation  is  roughly  the  same. 
Note  that  recorder  accuracy  is  1  second  while  latency  accuracy  is  in  milliseconds.  This  can  lead 
to  negative  overhead  values. 
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Overall,  HLA  and  JTIDS  system  overhead  is  a  small  contribution  to  message  latency. 


HLA+System  Overhead  for  Ref  Net  Messages 
(Response  Time  -  Send  Time  -  Reported  Latency) 

(Case:  10  sec  requests  on  Ref  Net;  10  sec.  Bulk,  All  position  updates) 


Figure  34  -  HLA  +  System  Overhead  for  Ref  Net  under  Minimal  Load 
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HLA  +  System  Overhead  for  Ref  Net  Messages 
(Response  Time  -  Send  Time  -  Reported  Latency) 
(Case:  41  step  net,  1/2  rate;  10  sec  updates  ALL,  Bulk) 
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Figure  35  -  HLA  +  System  Overhead  for  Ref  Net  under  Moderate  Load 
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8.2  Review  of  Main  Load  Runs 


Recall  that  the  main  PC  load  experiments  were  summarized  in  Table  12.  This  section  presents 
charts  and  analyses  for  these  runs.  Much  of  the  information  presented  will  also  be  relevant  to 
analyses  presented  in  the  sections  that  follow. 


8.2.1  Review  of  Charts  and  Simulation  Behavior 

For  each  experiment  load  run  we  produced  a  “standard”  set  of  charts.  The  first  two  charts  are  the 
Sim/Real  Ratio  and  Reference  Net  Latency  plotted  against  wall  clock  run  time.  Following  these 
charts  is  a  chart  of  cumulative  MSG  SEND  (MS)  and  MSG  RCVD  (MR)  interactions 
positioned  above  a  chart  of  Cumulative  Expected  and  Relay  counts.  The  remaining  two  charts 
are  time  moving  averages  (TMA)  of  these  four  values.  These  charts,  individually  and  taken 
together,  can  tell  an  interesting  story.  We  start  by  comparing  the  charts  shown  in  Figure  36. 


Figure  36  -  Sim/Real  Chart  Compared  with  Cumulative  MS  and  MR  Counts 
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Figure  36  compares  a  chart  of  Sim/Real  Ratio  (SRR)  to  cumulative  counts  of  MS  and  MR.  The 
Sim/Real  Ratio  shown  is  for  the  highest  (full)  load  on  the  41  Net  Step  case.  SRR  quickly  starts 
to  drop  when  the  first  two  loads  are  introduced,  and  drops  to  a  low  of  approximately  0.54.  The 
behavior  of  the  cumulative  chart  for  MS  and  MR  follows  the  SSR  curve.  First  note  that  the 
cumulative  curve  for  MS  closely  follows  the  expected  ideal  curve  for  the  driving  function  (4 1 
Net  Step  requests).  However,  the  curve  for  cumulative  MR  quickly  departs  from  the  MS  curve 
and  is  not  at  all  close  to  the  ideal  curve.  As  shown,  the  gaps  between  the  MS  and  MR  curves  can 
be  interpreted  in  two  ways  -  horizontal  and  vertical.  Horizontal  gaps  indicate  the  amount  of  time 
that  responses  are  behind  the  requests.  Vertically,  the  gaps  tell  how  many  messages  are  lagging 
behind  in  responses.  The  overall  magnitude  of  the  load  is  indicated  by  the  amount  of  time  that 
SRR  spends  below  the  ideal  value  of  1.0.  Clearly  the  full  41  Net  Step  load  is  too  high  to  be 
useful  with  JSAF  because  of  the  high  latency  delays  and  the  position  inaccuracies  that  will  occur. 

Figure  37  validates  this  conclusion.  Under  no  load,  the  Reference  Net  has  an  average  latency  of 
1.2  seconds.  Under  the  highest  load  portion  (middle  of  the  run)  the  full  41  Net  Step  load,  the 
moving  average  latency  of  the  Reference  Net  goes  up  to  15  seconds  -  over  10  times  the  no-load 
value.  The  other  charts  in  the  sequence  tell  a  similar  story. 
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Figure  37  -  Ref  Net  Latency  for  No  Load  vs.  Full  41  Net  Step  Load 
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Figure  38  -  Comparison  of  Cum  MS  and  MR  vs.  Cum  E  and  R  counts 
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Figure  38  compares  the  charts  for  cumulative  MS  and  MR  with  charts  for  cumulative  E  and  R 
counts.  We  explored  the  left  chart  before.  The  chart  for  cumulative  E  and  R  deviates  from  the 
ideal  step  response  curve.  The  shape  shows  break  points  in  the  same  places  that  the  SRR  curve 
changes. 

More  interesting  are  the  TMA  charts  for  MS  and  MR  and  E  and  R  shown  below  in  Figure  39.  At 
first  the  shape  of  the  TMA(MR)  and  TMA(E)  and  TMA(R)  curves  surprised  us.  The  TMA(MS) 
curve  however  reflected  the  real-time  stepped  load  of  requests  that  were  being  applied.  We 
concluded  that  the  large  “bump”  shown  for  TMA(MR),  TMA(E)  and  TMA(R)  is  a  result  of  the 
extended  drop  in  SRR  and  messages  therefore  being  queued  up  in  the  system  -  primarily  in  the 
simulation  schedule  queue.  Once  the  driven  load  turns  off,  then  the  JTIDS  simulation  rather 
quickly  flushes  out  the  pent-up  transmission  requests  that  generate  a  flurry  of  transmission 
responses. 


Figure  39  -  Comparison  of  TMA  charts  of  MS  and  MR  vs.  E  and  R 


Obviously,  the  full  41  Net  Step  load  is  too  high  for  this  computer.  The  next  section  provides  a 
comparison  across  the  loads  that  were  used  and  an  analysis  of  what  can  be  learned  from  the  data. 
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8.2.2  Cross  Load  Run  Analysis 

This  section  looks  across  the  experimental  data  to  explore  patterns  that  may  be  extracted.  Figure 
40  shows  a  chart  that  compares  the  average  and  min/max  values  of  the  Reference  Net  latency  for 
each  of  the  load  runs  performed  on  the  Main  Test  PC.  Each  case  shows  the  average  value  of 
Expected  +  Relay,  e.g.,  <79>,  and  the  worst-case  SRR  in  the  run,  e.g.,  [.883].  Notice  that  as  the 
SRR  drops,  the  average  Ref  Net  Latency  increases  rather  smoothly  to  about  4  times  the  no-load 
value,  however  the  max  latency  values  spread  by  almost  a  factor  of  7  to  8.  While  there  is  some 
correlation  between  SRR  and  <E+R>,  there  are  unexpected  differences  between  the  SRR  values 
for  the  41  Net  cases  and  the  81  Net  cases  compared  to  <E+R>. 


Table  15  and  Table  16  shed  some  light  on  the  measured  differences.  Each  table  lists  the  net  sets 
within  each  net  group  and  analyzes  the  number  of  stationary  and  moving  sources  within  each  net 
set.  82.5%  of  the  sources  in  the  41  Net  Group  are  moving  compared  to  60%  of  the  sources  in  the 
81  Net  Group.  In  the  41  Net  Group,  79%  of  the  moving  sources  are  fast  movers,  e.g.,  airborne 
platforms.  In  the  81  Net  Group,  65%  of  the  moving  sources  are  fast  movers.  65%  of  all  sources 
in  the  41  Net  Group  are  fast  movers,  whereas  only  38.8%  of  all  sources  in  the  81  Net  Group  are 
fast  movers.  The  tables  also  list  the  number  of  platfonn  destinations  in  each  net  set,  and  total  the 
destinations  within  each  Net  Group.  The  two  Net  Groups  have  roughly  the  same  number  of  total 
destinations,  4080  for  the  81  Net  Group  versus  3600  for  the  41  Net  Group.  Since  the  41  Net 
Group  has  a  much  higher  percentage  of  moving  sources,  transmission  requests  will  invoke  FPPS 
calculations  much  more  often  than  for  the  81  Net  Group. 


Furthermore,  when  the  scenario  and  nets  for  the  8 1  Net  Group  were  designed,  a  high  percentage 
of  moving  platforms  were  dropped  to  reduce  the  number  of  end-points  in  each  net,  many  of  these 
were  FI 5s  and  FI 8s.  Therefore,  the  81  Net  Group  is  more  likely  to  have  a  higher  percentage  of 
stationary  destinations,  which  will  reduce  propagation  calculations  further. 
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Table  15  -  Analysis  of  41  Net  Group  Sources 

41  Net  Group  Sources 


Net  Set 

Stationary 

Moving 

Fast 

Slow 

#  Dests  per  Net 

PPLI_B  No  Relay 

2 

8 

8 

0 

86 

PPLI_B  1  Relay 

1 

9 

8 

1 

86 

MM  No  Relay 

2 

8 

5 

3 

94 

MM  1  Relay 

2 

8 

5 

3 

94 

Totals 

7 

33 

26 

7 

3600 

%  of  Total 

17.5% 

82.5% 

65.0% 

17.5% 

%  of  Movers 

79% 

21% 

Table  16  -  Analysis  of  81  Net  Group  Sources 

81  Net  Group  Sources 


Net  Set 

Stationary 

Moving 

Fast 

Slow 

#  Dests  per  Net 

PPLIB  No  Relay 

13 

7 

5 

2 

56 

PPLIB  1  Relay 

3 

17 

12 

5 

48-56 

MM  No  Relay 

14 

6 

6 

0 

48 

MM  1  Relay 

2 

18 

8 

10 

48 

Totals 

32 

48 

31 

17 

4080 

%  of  Total 

40.0% 

60.0% 

38.8% 

21.3% 

%  of  Movers 

65% 

35% 

Also  consider  that  the  request  rate  for  nets  with  1  relay  is  half  that  for  nets  with  no  relays.  In  the 
41  Net  Group,  all  net  sets  have  approximately  the  same  number  of  moving  sources  for  nets  with 
and  without  relays.  On  the  other  hand,  8 1  Net  Group  sources  without  relays  have  half  as  many 
movers  than  sources  with  relays.  This  implies  that  transmissions  on  the  relay  nets  in  the  8 1  Net 
Group  will  likely  be  higher  loads. 

These  observations  begin  to  explain  some  of  the  differences  observed  in  the  cross-run 
comparisons  and  the  results  charted  in  Figure  40.  The  analyses  begin  to  suggest  the  possibility 
of  using  the  experiment  data  to  “predict”  loading  for  different  scenarios  and  net  designs  being 
driven  by  specific  patterns  of  transmission  requests.  While  tantalizing,  this  is  outside  the  scope 
of  this  experimentation  task. 

The  charts  that  follow  complete  this  portion  of  the  cross  run  analysis.  Figure  presents  latency 
data  for  selected  net  in  the  41  Net  Step  Flalf  Rate  run.  Latencies  are  shown  side-by-side  for  the 
PPLI_B  and  Mission  Management  (MM)  nets  with  no  relay  and  those  that  have  1  relay.  At  the 
bottom  of  the  figure,  the  SSR  and  Ref  Net  latency  charts  are  shown  for  reference.  The  41  Net 
Step  Flalf  Rate  case  is  a  moderate  load  on  the  Main  Test  PC;  the  SRR  falls  to  slightly  less  than 
0.9  for  less  than  400  seconds  during  the  peak  load  within  this  loading  case.  The  top  four  charts 
in  Figure  correspond  to  Steps  1,  2,  3  and  4  of  the  stepped  load  run.  The  right-hand  charts  show 
latency  for  a  relayed  destination,  e.g.,  reached  by  a  relayed  transmission.  The  left-hand  charts 
show  latency  for  direct  transmission  from  sources  to  destinations. 
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PPLI_B  Latency  without  Relay  (Net:  18,  Dest:  5470) 


PPLIB  Latency  with  Relay  (Net:  37,  Dest:  19877) 
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Figure  41  -  Comparison  of  Selected  Results  for  41  Net  Step  Half-Rate  Load 


Notice  that  the  Mission  Management  nets  have  lower  latency.  This  is  because  they  have  higher 
throughput  than  the  PPLI  B  nets.  The  latencies  for  all  the  nets  shown  are  good  with  variations 
that  are  within  reasonable  limits. 


The  next  set  of  charts  is  for  the  41  Net  Step  Full  Rate  case.  These  charts  are  shown  in  Figure  42 
and  are  formatted  as  described  for  the  41  Net  Step  Half  Rate  case  in  Figure  41.  The  immediate 
observation  for  this  case  is  that  the  processor  is  loaded  to  the  point  where  latency  is  clearly  too 
long  by  a  huge  factor.  The  next  observation  is  that  there  is  a  dramatic  difference  between  the 
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lower  throughout  PPLI  B  nets  and  the  higher  throughput  Mission  Management  nets.  Latency 
varies  by  a  factor  of  amount  10  between  these  net  types.  Also,  for  this  load  case,  the  relay  nets 
show  a  lower  latency.  Since  the  SRR  drops  to  less  than  0.5  for  over  80%  of  the  run,  this  load  is 
clearly  unreasonable. 


PPLI_B  Latency  without  Relay  (Net:  18,  Dest:  5470) 
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Figure  42  -  Comparison  of  Selected  Results  for  41  Step  Full  Rate  Load 
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8.4  Position  Update  Sensitivity  Analysis 


To  evaluate  the  sensitivity  of  loading  to  position  update  rates,  we  performed  a  total  of  eight  load 
runs  as  detailed  in  Table  13  in  Section  7.0.  The  41  Net  Step  Half-Rate  load  was  chosen  as  a 
“moderate”  load,  and  the  81  Net  Step  Full-Rate  was  chosen  as  a  “high”  load.  We  initially  drove 
each  request  load  case  with  position  update  rates  of  14,  10  and  6  seconds.  The  10-second  rate  is 
our  standard  rate,  and  the  other  rates  are  40%  higher  and  lower.  After  comparing  SRR  results 
across  the  runs,  we  decided  to  perfonn  two  more  runs  on  the  41  Net  case,  one  at  a  rate  of  8 
seconds  and  another  at  a  faster  rate  of  2  seconds.  We  decided  to  do  no  further  update  rate  tests 
on  the  8 1  Net  case  because  the  8 1  Net  load  was  too  high. 


The  results  of  the  sensitivity  runs  are  shown  in  Figure  43.  For  the  41  Net  Step  Half  Rate  case, 
there  is  very  little  difference  between  the  14s  and  18s  update  rate.  This  seems  reasonable  since 
the  lower  update  rate  (longer  interval)  will  induce  fewer  FPPS  calculation.  Similar  results  are 
expected  for  much  higher  update  rates  since  updates  between  transmission  requests  should  not 
increase  the  FPPS  load  on  the  system.  This  is  not  apparent  in  the  figure,  although  there  is  a  hint 
of  this  for  the  41  Net  Full  Rate  case.  The  higher  sensitivity  of  the  81  Net  Full  Rate  case  to 
position  update  rate  is  a  little  surprising.  We  suspect  that  high  load  on  the  system  for  this  case 
makes  this  load  case  more  sensitive  to  the  update  rate.  We  feel  that  any  load  that  drives  the  SRR 
to  below  0.8  should  be  avoided. 
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8.5  Cross-PC  Run  Analysis 


Data  for  the  cross-PC  runs  was  shown  in  Table  14  in  Section  7.0.  Five  additional  load 
experiments  were  perfonned  to  gather  data  to  analyze;  two  runs  were  completed  on  the  Test 
Laptop,  and  three  runs  were  done  on  the  Custom  PC.  The  SRR  for  these  runs  is  compared  in 
Figure  44. 
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Figure  44  -  Comparison  of  Sim/Real  Ratio  in  Cross-PC  Experiments 


Only  the  41  Net  Step  Quarter  and  Half  rate  runs  were  completed  on  the  Test  Laptop  with 
unacceptable  results.  This  Laptop  PC  is  an  old  850  MHz  P3  machine  with  only  256  MB  of 
RAM.  Attempts  to  run  the  41  Net  Step  Full  rate  were  aborted  due  to  excessive  time  and  abysmal 


SRR. 


The  comparison  between  the  Main  Test  PC  and  the  Custom  Desktop  PC  is  more  interesting.  The 
Main  Test  PC  is  a  Dell  Desktop  that  is  less  than  a  year  old  and  has  a  2.8  GHz  P4  with  1  GB 
RAM  running  Windows  XP  Pro  SP2.  The  Custom  Desktop  is  over  two  years  old  and  is  a  3  GHz 
P4  machine  with  2  GB  RAM  running  Windows  2K  SP4.  We  believe  that  two  factors  account  for 
the  differences  in  loading  between  these  PCs:  1)  machine  age  -  we  suspect  that  the  newer 
machine  has  a  higher  speed  memory  bus,  and  2)  XP  versus  2K.  PSI  has  seen  a  slight  PC 
performance  edge  in  Windows  XP  Pro  over  Windows  2K.  The  additional  memory  in  the 
Custom  PC  apparently  did  not  make  a  difference  compared  to  other  factors.  Also,  although  an 
attempt  was  made  to  close  all  applications  loaded  into  the  Windows  desktop  tray  after  re¬ 
booting,  there  may  have  been  some  drivers  consuming  sources  in  the  Custom  PC. 
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Note  that  the  Custom  PC  has  a  much  higher  resolution  screen  and  a  high-end  graphics  card. 
Since  there  was  a  possibility  that  screen  resolution  could  be  a  factor,  we  re-prepared  the  JTIDS 
simulation  for  operation  in  Hardware  Graphics  Mode  (HWG).  This  mode  of  operation  made 
essentially  no  difference  in  measured  SRR.  Note  that  the  41  Net  Step  Full  rate  run  with  HWG 
was  aborted  since  there  was  no  apparent  difference  with  respect  to  the  non-hardware  graphics 
mode. 


8.6  Long  Stability  Run 

There  is  a  distinct  chance  that  the  JTIDS  simulation  could  be  used  to  support  JSAF  and  other 
similar  large  forces  simulations  over  an  extended  period  of  time.  Therefore,  we  decided  to  run 
an  experiment  for  an  extended  period  of  10  hours.  All  prior  runs  were  no  longer  than  60 
minutes,  and  most  runs  were  less  than  40  minutes.  We  drove  traffic  requests  to  all  41  nets  in  the 
41  Net  Group  at  half  rate  while  sending  position  updates  every  10  seconds. 

After  10  hours  the  long  run  was  manually  ended.  Over  500,000  position  updates  were  sent  to  the 
system,  and  over  46,000  transmission  requests  were  sent.  The  long  run  was  an  excellent  test  of 
the  stability  of  the  GIESim  JTID  Simulation.  This  is  the  first  time  that  the  simulation  was  left 
running  this  long  or  had  this  much  traffic  sent  through  it.  Our  conclusion  is  that  there  are  neither 
apparent  memory  leaks  nor  other  accumulating  errors  in  the  simulation,  and  that  the  GIESim 
JTIDS  simulation  should  run  much  longer. 


8.7  Message  Loss  Analysis 

For  each  of  the  load  runs  we  computed  the  difference  between  the  requests  sent  and  the 
responses  received.  The  difference  is  considered  a  loss  of  messages.  In  general  we  attempted  to 
design  the  networks  and  chose  destinations  for  transmission  such  that  network  connectivity 
should  always  be  good.  However,  the  run  time  of  the  original  Korona  scenario  was  about  20 
minutes,  after  which  platforms  loop  back  on  their  movement  paths.  After  a  few  movement 
cycles,  the  original  synchronization  between  moving  platforms  is  lost.  While  this  should  not 
introduce  any  operational  errors,  the  scenario  dynamics  are  different  and  there  is  a  possibility  of 
platforms  being  out  of  position  with  respect  to  the  initial  scenario. 

Across  all  experiments,  the  message  loss  was  generally  less  than  2%.  The  overloaded  laptop  lost 
3%  of  messages.  6%  of  messages  were  lost  in  the  long  run.  Overall,  we  believe  these  are  very 
reasonable  loss  numbers. 
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9.0  Future  Work  Possibilities 


During  any  experiment,  observations  arise  that  can  suggest  other  possibilities  for  work  and 

potential  modifications  and  enhancements  of  tools  and  experimental  design.  This  section  briefly 

captures  these  ideas  for  future  consideration. 

•  All  Destination  Address:  The  original  planning  and  design  for  GIESim/JSAF 
interoperability  considered  the  idea  of  using  a  destination  address  of  zero  to  signify  that  all 
receivers  of  the  message  should  respond.  Some  of  the  code  was  put  into  JTIDS  to  support 
this,  and  the  GIESim/JSAF  team  never  exploited  the  capability  because  of  the  high  volume 
of  HLA  response  traffic  possible,  and  because  we  never  identified  an  operational  case  that 
called  for  this  feature. 

•  Other  Access  Modes:  Link- 16  radios  support  Dedicated,  Contention  and  Dedicated  Slot  Re¬ 
use  Access  Modes.  The  JTIDS  simulation  supports  these  modes  in  addition  to  new  modes 
developed  by  PSI.  The  GIESim  version  of  JTIDS  also  supports  these  modes  although  they 
have  not  been  extensively  tested,  and  only  the  Dedicated  Access  Mode  was  used  during 
experimentation. 

•  Use  of  SAR:  The  GIESim  JTIDS  Simulation  supports  Segmentation  and  Re-assembly 
(SAR)  of  messages  that  will  not  fit  into  the  capacity  of  an  operational  net.  SAR  complicates 
loading  since  extra  messages  are  generated  that  are  not  included  in  the  existing  metrics.  SAR 
was  not  tested  during  experimentation. 

•  SRR  Request/Response:  Since  the  Sim/Real  ratio  (SRR)  is  a  critically  important  load 
factor;  the  existing  GIESim  HLA  interactions  could  be  used  with  special  parameters  to 
request  a  response  that  contains  the  current  SRR. 

•  More  Complex  Networks:  For  some  applications  with  JSAF  and  similar  simulations  more 
complex  networks  with  more  relays  may  be  required.  PSI  performed  two  experiments  on  the 
original  Korona  networks  that  had  12+  relays.  We  decided  to  scrap  this  data  due  to  the 
extremely  high  SRR  incurred  and  extensive  latency  and  message  delays.  We  think  that 
operational  nets  should  use  no  more  than  3-4  relays.  Load  experiments  on  nets  with  2,  3  and 
4  relays  would  provide  more  data  for  sizing  potential  operations  with  JSAF,  etc. 

•  Machine  Synchronization  Messages:  During  experimentation  we  frequently  waited  for  the 
GIESim  JTIDS  simulation  to  initialize  before  we  started  our  load  generators  and  recorders. 
The  same  coordination  is  needed  in  operation  with  JSAF  and  other  large  forces  simulations. 
The  idea  developed  to  use  existing  GIESim  HLA  interactions  with  specific  parameters  to 
trigger  the  start  of  other  simulations  and  to  synchronize  them.  To  some  extent  this  has  been 
used  in  the  past,  and  could  be  extended  to  our  newer  tools,  such  as  the  HLA  Recorder. 

•  HLA  Time  Management:  As  the  size  and  complexity  of  scenarios  and  networks  increase,  a 
point  will  be  reached  where  any  one  computer  will  not  be  able  to  attain  sufficient  SSR  to 
achieve  required  accuracy.  We  may  want  to  consider  introduction  of  some  form  of  cross 
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simulation  time  management  to  keep  the  simulation  clocks  closely  aligned.  Of  course  this 
would  drop  both  JSAF  and  JTIDS  out  of  the  realm  of  real  time  operation  if  message  traffic 
were  very  high.  This  would  require  sizable  modification  of  JTIDS. 

•  Update  Filtering:  The  option  to  filter  updates  that  do  not  change  position  locations  would 
eliminate  some  computation  overhead  associated  with  updating  the  platform  and  link 
databases  in  JTIDS.  This  would  be  an  easy  change  to  JTIDS. 

•  High  Performance  Computers:  Higher  load  conditions  will  require  higher  perfonnance 
computers.  Possibilities  include  new  dual-core  PCs,  near-term  multi-core  PCs,  and 
Massively  Parallel  Processor  (MPP)  machines.  The  latter  is  preferred  because  of  the  likely 
need  to  support  large  scenarios  and  high  network  traffic.  Based  on  the  eventual  need  to  run 
very  large  JTIDS  simulations  very  fast,  and  the  fact  that  the  platforms  within  the  simulation 
are  coupled  due  to  their  potential  interactions  in  a  shared  electromagnetic  space,  PSI  has 
concluded  that  the  most  effective  processing  solution  is  a  single-OS  MPP  machine  with 
shared  memory. 

Processing  requirements  analysis  done  on  another  project  determined  that  a  single,  very  large 
simulation  could  not  run  on  a  single  PC  and  meet  the  desired  response  requirements. 
Alternatives  that  are  potentially  available  include: 

•  Beowulf  clusters:  Beowulf  clusters  are  potentially  good  for  simulations  of  very  loosely 
coupled  systems.  For  simulations  that  have  a  potentially  high  degree  of  coupling 
however  (not  embarrassingly  parallel),  such  as  the  simulation  of  a  MTN,  the  inter¬ 
processor  network  latency  becomes  a  major  limitation  due  to  the  number  of  messages 
that  must  be  exchanged  to  synchronize  the  simulation  state  -  particularly  of  the 
electromagnetic  environment. 

•  HLA  or  DIS  Distributed  Computing  Environment:  HLA  and  DIS  are  typically  used 
to  interconnect  disparate  simulations.  The  distance/intercommunication  loss  for  an 
HLA/DIS  approach  is  estimated  to  be  10  to  100  greater  than  for  a  Beowulf  cluster. 

While  HLA  is  acceptable  between  JSAF  and  JTIDS,  it  is  much  too  slow  to  couple 
multiple  computers  aggregated  to  support  computation  of  JTIDS  networking. 

•  MPP  machine:  In  a  massively  parallel  processor  supercomputer  with  a  single-OS,  inter¬ 
process  communication  can  be  exceptionally  fast,  and  shared  memory  architectures  can 
support  an  essentially  common  electromagnetic  environment.  Furthermore,  GSS  can 
make  much  better  use  of  parallel  threads  (p-threads)  on  an  MPP  machine.  PSI  has  met 
with  several  parallel  processing  companies  to  further  its  support  for  parallel  processing  in 
GSS  simulations.  Several  companies  including  IBM  and  SGI  have  found  the  approach  to 
building  software,  e.g.,  simulations,  for  parallel  processing  created  by  PSI  to  be  highly 
effective,  a  very  good  fit  for  the  MPP  hardware,  and  one  that  they  think  is  on  the  leading 
edge.  Of  the  MPP  machines  that  PSI  has  looked  at,  the  SGI  machines  seem  to  be  an 
excellent  first  choice  for  JTIDS  Planning  and  Validation.  PSI  is  currently  in  favor  of  the 
SGI  Altix  3000  family  of  super  computers.  These  computers  have  a  hardware 
architecture  that  is  optimal  for  the  type  of  computing  required  for  JTIDS  networking. 
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10.0  Conclusions 


Our  overarching  conclusion  is  that  the  Sim/Real  ratio  (SRR)  is  the  best  indicator  of  system  load 
on  the  GIESim  JTIDS  simulation.  Scenario  sizes  on  the  order  of  136  Link- 16  platforms  can 
easily  be  supported  and  external  position  update  rates  of  every  10  seconds  are  acceptable,  which 
is  typical  of  what  is  expected  from  JSAF.  The  GIESim  JTIDS  simulation  can  be  used  for 
experiments  of  extended  duration  of  many  hours. 

Network  loading  is  more  complex  and  less  predictable  than  originally  anticipated  and  is  highly 
scenario  dependent.  Generally,  SRR  should  be  0.8  or  higher  for  the  majority  of  experimentation 
on  a  machine  of  comparable  performance  to  our  Main  Test  PC.  For  networks  of  the  size  and 
complexity  of  our  Net  Groups  used  in  testing,  average  request  rates  on  any  one  operational  net 
should  be  much  less  than  half  (preferably  a  quarter)  of  the  response  time  or  throughput  for  the 
net. 

After  analysis  of  experimentation  results,  and  in  retrospect,  we  believe  the  set  of  steady  request 
rates  that  we  used  were  higher  than  would  typically  be  experienced  in  Command  and  Control 
(C2)  operations,  which  are  more  likely  to  be  more  bursty.  C2  operations  are  the  primary  focus  of 
GIESim/JSAF.  An  exception  to  this  may  be  in  the  handling  of  surveillance  traffic  where  high 
volumes  of  track  data  may  be  exchanged,  which  implies  a  potential  high  request  rate  on  high 
throughput  nets.  As  experience  with  JSAF  unfolds,  additional  experimentation  may  be 
identified. 

Because  positions  updates  and  transmission  requests  are  happening  in  real  time,  the  need  to  keep 
SRR  high  is  very  important.  As  SRR  drops  much  below  0.9,  both  transmission  latency  and 
position  accuracy  at  the  time  of  transmission  become  more  and  more  inaccurate.  The  chart 
shown  in  Figure  45  provides  an  overview  of  these  relationships. 
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Figure  45  -  Relationships  of  Position  Accuracy  and  Latency  versus  System  Load 
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Additional  conclusions  are  listed  below: 


■  The  experimentation  load  test  configuration  can  be  used  to  assess  support  and  capacity 
for  planned  interoperability  runs  with  JSAF: 

o  Use  the  Link- 16  Planning  Tool  to  design  nets  and  scenarios  of  anticipated 
complexity. 

o  Use  the  test  configuration  to  exercise  the  planned  operation. 

■  During  live  operations  with  JSAF,  the  JTIDS  Sim  Clock  display  and  dynamic  Sim/Real 
ratio  display  can  be  used  to  assess  performance  of  JTIDS.  The  JTIDS  Monitor  and  HLA 
Recorder  can  be  used  to  collect  data  for  post  analysis  following  an  operation. 

■  Any  anomalous  behaviors  between  JSAF  and  JTIDS  can  be  explored  by  playing  back 
recorded  HLA  traffic  of  ENTITY  STATE  and  MSG  SEND  messages  into  JTIDS  to 
search  for  root  causes  that  could  include: 

o  JTIDS  or  JSAF  operational  problem 

o  Real  time  slips  in  either  JSAF  or  JTIDS 

o  Potential  network  design  issues,  or  incorrect  operational  use  of  (or  expectations 
of)  operational  nets. 


To  a  very  large  extent  the  GIESim  experimentation  work  was  highly  interesting,  challenging  and 
thought  provoking.  Preliminary  experiments  lead  to  refinements  of  GIESim  JTIDS  that  make  it 
more  robust  and  much  more  efficient  for  operational  use.  These  early  experiments  also  lead  to 
much  more  powerful  and  effective  test  tools  for  use  with  GIESim  JTIDS  and  JSAF.  Over  26 
hours  of  final  experimentation  has  resulted  in  the  collection  of  over  100  MB  of  data  that  can  still 
be  mined  for  additional  information.  The  process  of  designing,  executing  and  analyzing  the 
experiments  has  lead  to  much  deeper  insights  into  GIESim  JTIDS  and  operational  considerations 
for  use  with  JSAF  and  other  similar  simulations. 

PSI  deeply  appreciates  the  GIESim  leadership  team  in  AFRL  Rome,  NY  for  this  opportunity  to 
experiment  with  GIESim  JTIDS.  We  sincerely  hope  that  GIESim  JTIDS  will  be  applied  to 
further  operations  with  JSAF  and  with  the  JSB-RD  team  in  Rome  and  with  USJFCOM  J9.  PSI 
is  committed  to  the  success  of  the  GIESim  Laboratory  and  looks  forward  to  an  on-going 
relationship  with  AFRL  and  the  other  GIESim/JSB-RD  team  members  in  Rome. 
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Appendix  A  -  Test  Networks 


A.1:  41  Net  Group  Details 

Table  17  -  41  Net  Set  Groups 


Target  Destinations 


Resp 

Words 

Msgs 

Test 

Net 

# 

# 

Time 

per 

per 

Unit  of 

D2 

Set 

Net  ID 

Type 

Source 

Dest's 

Relay 

(sec) 

TJ 

Msg 

unit  time 

Time 

Packing 

D1 

Relay  Dest 

Relay 

Ref  Net  162  REFNET  GH_REF_SRC  1  0  10  J15  4  12  12  P4  GH_REF_DEST 


Set  1 

KORONA_REF_RELAY.NET 

16 

PPLI  B 

ABRLY  2  1 

86 

0 

10 

J2/13 

6 

1 

12 

P2DP 

ABRLY  3  1 

17 

PPLIB 

ABRLY22 

86 

0 

10 

J2/13 

6 

1 

12 

P2DP 

JSTARS 2 1 

18 

PPLI  B 

ABRLY  2  3 

86 

0 

10 

J2/13 

6 

1 

12 

P2DP 

E3  2  4 

33 

PPLIB 

E3 2 1 

86 

0 

10 

J2/13 

6 

1 

12 

P2DP 

ABL22 

34 

PPLI  B 

E3  2  2 

86 

0 

10 

J2/13 

6 

1 

12 

P2DP 

JLENS  2  2 

35 

PPLIB 

E3 2 3 

86 

0 

10 

J2/13 

6 

1 

12 

P2DP 

GHAWK29 

107 

PPLI  B 

E2C  3  1 

94 

0 

10 

J2/13 

6 

1 

12 

P2DP 

F15F  2  5 

108 

PPLI B 

E2C32 

94 

0 

10 

J2/13 

6 

1 

12 

P2DP 

E2C 3 3 

76 

PPLI  B 

THAADTOC  2  3 

86 

0 

10 

J2/13 

6 

1 

12 

P2DP 

ABRLY  3  1 

77 

PPLI B 

TH  AADT  OC 2 4 

86 

0 

10 

J2/13 

6 

1 

12 

P2DP 

ABRLY 2 1 

Set  2  - 1  Relay  Each 
KORONA_REF_RELAY.NET 

109 

PPLIB 

E2C33 

94 

1 

10 

J2/13 

6 

1 

12 

P2DP 

E2C 1 2 

ABRLY31 

110 

PPLI  B 

E2C  3  4 

94 

1 

10 

J2/12 

6 

1 

12 

P2DP 

SHIP  2  1 

ABRLY  2  3 

111 

PPLI  B 

EP3  3  1 

94 

1 

10 

J2/13 

6 

1 

12 

P2DP 

SHIP  3  7 

E3  2  2 

36 

PPLI B 

E3 2 4 

86 

1 

10 

J2/13 

6 

1 

12 

P2DP 

GHAWK 2 1 1 

GHAWK210 

37 

PPLI  B 

RJ  2  1 

86 

1 

10 

J2/13 

6 

1 

12 

P2DP 

SHIP  3  1 

GHAWK  2  9® 

38 

PPLIB 

RJ 2 2 

86 

1 

10 

J2/13 

6 

1 

12 

P2DP 

SHIP 3 2 

E2C 3 3 ® 

39 

PPLI  B 

JSTARS  2  1 

86 

1 

10 

J2/13 

6 

1 

12 

P2DP 

SHIP  3  11 

E2C  3  3® 

130 

PPLIB 

E2C 1 1 

95 

1 

10 

J2/13 

6 

1 

12 

P2DP 

E2C32 

GHAWK 2 10  ®  (N) 

74 

PPLI  B 

THAADTOC  2  1 

86 

1 

10 

J2/13 

6 

1 

12 

P2DP 

SHIP  1  3 

ABRLY  2  2 

136 

PPLIB 

SHIP 3 1 

86 

1 

10 

J2/13 

6 

1 

12 

P2DP 

JTAGS23 

ABRLY21  (B) 

Set  3  -  No  Relays 
KORONA_REF_RELAY.NET 

112 

MISSION MGT 

E2C 3 1 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

THAADT  OC 2 3 

113 

MISSION  MGT 

E2C  3  2 

79 

0 

10 

J9/10 

4 

6 

12 

P2DP 

F18  1  6 

114 

MISSION  MGT 

E2C  3  3 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

SHIP  3  7 

115 

MISSION MGT 

E2C34 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

SHIP31 

101 

MISSION  MGT 

SHORAD  2  1 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

JLENS  2  2 

119 

MISSION MGT 

SHIP 1 1 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

E2C 1 1 

137 

MISSION  MGT 

SHIP  3  1 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

E2C  3  2 

91 

MISSION MGT 

JSTARS 2 1 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

RJ 2 2 

125 

MISSION  MGT 

SHIP  1  4 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

PATRIOTIC  2  4 

85 

MISSION MGT 

PATRIOTICC 2 4 

94 

0 

10 

J9/10 

4 

6 

12 

P2DP 

E3 2 1 

Set  4  - 1  Relay  Each 
KORONA_REF_RELAY.NET 

28 

MISSION MGT 

SHIP 3 2 

94 

1 

10 

J9/10 

4 

6 

12 

P2DP 

PATRIOTICC 2 3 « 

E2C  3  4 (B) 

89 

MISSION  MGT 

E3  2  3 

94 

1 

10 

J9/10 

4 

6 

12 

P2DP 

JTAGS  2  4® 

ABRLY  2  1  (B) 

92 

MISSION MGT 

JSTARS 2 2 

94 

1 

10 

J9/10 

4 

6 

12 

P2DP 

JTAGS  2  3  ® 

ABRLY21  (B) 

96 

MISSION  MGT 

ABL  2  4 

94 

1 

10 

J9/10 

3 

8 

12 

P2DP 

SHIP  3  1  ® 

E2C  3  4 (B) 

104 

MISSION  MGT 

THAADTOC  2  3 

94 

1 

10 

J9/10 

4 

6 

12 

P2DP 

JTAGS  2  2® 

ABRLY  2  1  (B) 

121 

MISSION  MGT 

SHIP  1  2 

94 

1 

10 

J9/10 

4 

6 

12 

P2DP 

PATRIOTIC  2  4  0 

E3 2 2  (B) 

133 

MISSION  MGT 

E2C  1  2 

94 

1 

10 

J9/10 

4 

6 

12 

P2DP 

E2C  3  4® 

E3  2  1  (B) 

123 

MISSION MGT 

SHIP13 

94 

1 

10 

J9/10 

4 

6 

12 

P2DP 

ABIFC1  ® 

JSTARS21  (B) 

95 

MISSION  MGT 

ABL  2  3 

94 

1 

10 

J9/10 

3 

8 

12 

P2DP 

CAC2S  2  3® 

ABRLY  2  1  (B) 

97 

MISSION MGT 

CRC 2 1 

94 

1 

10 

J9/10 

3 

8 

12 

P2DP 

RJ 2 1 ® 

ABRLY 3 1  (B) 
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A.2:  81  Net  Group  Details 

Table  18  -  81  Net  Set  Groups  -  Part  1 


Target  Destinations 


Resp 

Words 

Msgs 

Net 

Net 

# 

# 

Time 

per 

per 

Unit  of 

D2 

Set 

Net  ID 

Type 

Source 

Dest's 

Relay 

(sec) 

TJ 

Msg 

unit  time 

Time 

Packing 

D1 

Relay  Dest 

Relay 

Ref  Net  100  REFNET  GH_REF_SRC  1  0  10  J15  4  12  12  P4  GH_REF_DEST 


PPLI_B  NO  RELAY 

KO  RONA_HALF_REF.NET 

37 

PPLI  B 

ABL  2  4 

56 

0 

10 

J2/13 

6 

1 

12 

P2DP 

SHORAD  22 

45 

PPLI  B 

ABIFC  1 

56 

0 

10 

J2/13 

6 

1 

12 

P2DP 

UAV  GND  2  1 

46 

PPLI  B 

ABIFC  2 

56 

0 

10 

J2/13 

6 

1 

12 

P2DP 

PATRIOTICC  2  4 

47 

PPLI B 

CRC 2 1 

56 

0 

10 

J2/13 

6 

1 

12 

P2DP 

ABRLY 3 1 

48 
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Table  19  -  81  Net  Set  Groups  -  Part  2 


Target  Destinations 

Net 

Set 

Net  ID 

Net 

Type 

Source 

# 

Dest's 

# 

Relay 

Resp 

Time 

(sec) 

TJ 

Words 

per 

Msg 

Msgs 

per 

unit  time 

Unit  of 
Time 

Packing 

D1 

D2 

Relay  Dest 

Relay 

MISSION_MGT  NO  RELAY 

KORONA_HALF_REF.NET 

60 

MISSION  MGT 

PATRIOTICC  2  3 

48 

0 

10 

J9/10 

4 

6 

12 

P2DP 

E2C  3  4 

60 
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48 

0 

10 

J9/10 

4 

6 

12 

P2DP 
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61 
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48 

0 

10 
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4 

6 

12 

P2DP 

E2C  1  1 

61 
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48 

0 

10 
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4 

6 

12 

P2DP 

ABRLY 1 1 

63 

MISSION  MGT 
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48 

0 

10 
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4 

6 

12 

P2DP 
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63 
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0 
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6 

12 

P2DP 
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65 
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48 
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10 
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6 

12 

P2DP 
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65 
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E3  2  4 

48 

0 

10 
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4 

6 

12 

P2DP 
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67 
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ABL 2 1 

48 

0 

10 
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3 

8 

12 

P2DP 
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67 
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ABL  2  1 

48 

0 
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12 
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70 
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48 

0 

10 
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8 

12 
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70 
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48 

0 

10 
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3 

8 

12 

P2DP 
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71 
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48 

0 

10 
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4 

6 

12 

P2DP 

ABRLY 2 2 

71 

MISSION MGT 
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48 

0 

10 
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4 

6 

12 

P2DP 

RJ 2 1 

72 

MISSION  MGT 

TAOM  2  2 

48 

0 

10 

J9/10 

4 

6 

12 

P2DP 

ABL  2  4 

72 

MISSION  MGT 

TAOM  2  2 

48 

0 

10 

J9/10 

4 

6 

12 

P2DP 

E2C  3  1 

74 

MISSION MGT 

THAADT  OC 2 1 

48 

0 

10 

J9/10 

4 

6 

12 

P2DP 

ABRLY 2 3 

74 

MISSION  MGT 

THAADTOC  2  1 

48 

0 

10 

J9/10 

4 

6 

12 

P2DP 

RJ  2  1 

75 

MISSION  MGT 

THAADTOC  2  2 

48 

0 

10 

J9/10 

4 

6 

12 

P2DP 

ABRLY  2  1 

75 

MISSION MGT 

THAADT  OC 2 2 

48 

0 

10 

J9/10 

4 

6 

12 

P2DP 

E2C 3 4 

MISSION  MGT  RELAY 

KORONA_HALF_REF.NET 

64 

MISSION  MGT 

E3  2  3 

48 

1 

10 

J9/10 

4 

6 

12 

P2DP 

JTAGS  2  4 

ABRLY  2  1 

66 

MISSION MGT 

JSTARS 2 2 

48 

1 

10 

J9/10 

4 

6 

12 

P2DP 

JTAGS 2 3 

ABRLY 2 1 

68 

MISSION MGT 

ABL 2 4 

48 

1 

10 

J9/10 

3 

8 

12 

P2DP 

SHIP 3 1 

E2C 3 4 

69 

MISSION  MGT 

CRC  2  1 

48 

1 

10 

J9/10 

3 

8 

12 

P2DP 

RJ  2  1 

ABRLY  3  1 

73 

MISSION  MGT 

SHORAD  2  1 

48 

1 

10 
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4 

6 

12 

P2DP 

CAC2S  2  1 

ABL  2  4 

82 

MISSIONMGT 

E2C 3 1 

48 

1 

10 

J9/10 

4 

6 

12 

P2DP 

JTAGS 2 2 

ABRLY 2 1 

83 
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E2C 3 2 

33 

1 

10 

J9/10 

4 

6 

12 
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TAOM 2 2 

ABIFC 1 

84 
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E2C  3  3 

48 

1 

10 
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4 

6 

12 
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48 

1 
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4 

6 
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JSTARS  2  2 

89 
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48 

1 

10 

J9/10 

4 

6 

12 

P2DP 
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ABRLY  2  2 

91 
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48 

1 
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4 

6 

12 

P2DP 

ABRLY 3 1 

E2C 1 2 

93 
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E2C  1  1 

48 

1 

10 

J9/10 

4 

6 

12 

P2DP 

SHIP  3  1 

E3  2  3 

95 

MISSION  MGT 

E2C  1  2 

48 

1 

10 

J9/10 

4 

6 

12 

P2DP 

E2C  3  4 

E3  2  1 

99 

MISSION MGT 

SHIP 3 1 

48 

1 

10 

J9/10 

4 

6 

12 

P2DP 

ABRLY 2 2 

E2C 3 4 

29 

MISSION  MGT 

SHIP  3  11 

48 

1 

10 
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4 

6 

12 

P2DP 

JTAGS  2  4 

E2C  3  2 

28 

MISSION MGT 

SHIP 3 7 

48 

1 

10 

J9/10 

4 

6 

12 

P2DP 

CRC 2 1 

ABRLY 3 1 

27 

MISSION  MGT 

SHIP  3  6 

48 

1 

10 

J9/10 

4 

6 

12 

P2DP 

CAC2S  2  3 

ABRLY  2  1 

26 
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1 
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4 

6 

12 
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25 
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4 

6 

12 

P2DP 

PATRIOTICC  2  4 

E2C  3  4 

13 
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48 

1 
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4 

6 

12 

P2DP 

RJ_2_2 
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Appendix  B  -  Run  Data 

B.1  Ref  Net  Only,  and  Position  Updates  every  10  sec  (Main  PC) 
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Figure  46  -  Ref  Net  Only  Sim/Real  Ratio 
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Figure  47  -  Ref  Net  Only  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 


Figure  48  -  Ref  Net  Only  Cumulative  MS  and  MR  Interactions 


Cumulative  Expected  (E),  Relays  (R)  and  E+R 


Figure  49  -  Ref  Net  Only  E,  R,  and  E+R 
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60  sec  TMA(MS)  and  TMA(MR) 


Figure  50  -  Ref  Net  Only  60  sec  TMA  (MS)  and  TMA  (MR) 


60  sec  TMA(E)  and  TMA(R) 


Figure  51  -  Ref  Net  Only  60  sec  TMA  (E)  and  TMA  (R) 
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B.2  41  Net  Step,  V4  Rate,  and  Position  Updates  every  10  sec  (Main  PC) 
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Figure  52  -  41  Net  Step  1/4  Rate  Sim/Real  Ratio 
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Figure  53  -  41  Net  Step  1/4  Rate  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  54  -  41  Net  Step  1/4  Rate  Cumulative  MS  and  MR  Interactions 
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Figure  55  -  41  Net  Step  1/4  Rate  Cumulative  E,  R  and  E+R 
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60  sec  TMA(MS)  and  TMA(MR) 
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Figure  56  -  41  Net  Step  1/4  Rate  60  sec  TMA(MS)  and  TMA  (MR) 
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Figure  57  -  41  Net  Step  1/4  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.3  41  Net  Step,  V2  Rate,  and  Position  Updates  every  10  sec  (Main  PC) 
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Figure  58  -  41  Net  Step  1/2  Rate  Sim/Real  Ratio 
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Figure  59  -  41  Net  Step  1/2  Rate  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  60  -  41  Net  Step  1/2  Rate  Cumulative  MS  and  MR  Interactions 
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Figure  61  -  Net  Step  1/2  Rate  Cumulative  E,  R,  and  E+R 
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60  sec  TMA(MS)  and  TMA(MR) 
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Figure  62  -  Net  Step  1/2  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  63  -  Net  Step  1/2  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.4  41  Net  Step,  Full  Rate,  and  Position  Updates  every  10  sec  (Main 
PC) 
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Figure  64  -  41  Net  Step  Full  Rate  Sim/Real  Ratio 
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Figure  65  -  41  Net  Step  Full  Rate  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  66  -  41  Net  Step  Full  Rate  Cumulative  MS  and  MR  Interactions 


Cumulative  Expected  (E),  Relays  (R)  and  E+R 
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Figure  67  -  41  Net  Step  Full  Rate  Cumulative  E,  R,  and  E+R 
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Figure  68  -  41  Net  Step  Full  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  69  -  41  Net  Step  Full  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.5  41  Net  Step,  Full  Rate,  and  Delta  Position  Updates  every  10  sec 
(Main  PC) 
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Figure  71  -  41  Net  Step  Full  Rate  Delta  Position  Update  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  72  -  41  Net  Step  Full  Rate  Delta  Position  Update  Cum  MS  and  MR  Interactions 


Cumulative  Expected  (E),  Relays  (R)  and  E+R 
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Figure  73  -  41  Net  Step  Full  Rate  Delta  Position  Update  Cum  E,  R,  E+R 
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Figure  74  -  41  Net  Step  Full  Rate  Delta  Position  Update  60  sec  TMA(MS)  and  TMA(MR) 


60  sec  TMA(E)  and  TMA(R) 


Figure  75  -  41  Net  Step  Full  Rate  Delta  Position  Update  60  sec  TMA(E)  and  TMA(R) 
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B.6  41  Net  Step,  V2  Rate,  and  Position  Updates  every  14  sec  (Main  PC) 
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Figure  76  -  41  Net  Step  1/2  Rate  14  sec  Position  Update  Sim/Real  Ratio 
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Figure  77  -  41  Net  Step  1/2  Rate  14  sec  Position  Update  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  78  -  41  Net  Step  1/2  Rate  14  sec  Position  Update  Cum  MS  and  MR  Interactions 
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Figure  79  -  41  Net  Step  1/2  Rate  14  sec  Position  Update  Cum  E,  R,  and  E+R 
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60  sec  TMA(MS)  and  TMA(MR) 


Figure  80  -  41  Net  Step  1/2  Rate  14  sec  Position  Update  60  Sec  TMA(MS)  and  TMA(MR) 
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Figure  81  -  41  Net  Step  1/2  Rate  14  sec  Position  Update  60  sec  TMA(E)  and  TMA(R) 
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B.7  41  Net  Step,  V2  Rate,  and  Position  Updates  every  6  sec  (Main  PC) 
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Figure  82  -  41  Net  Step  1/2  Rate  6  sec  Position  Update  Sim/Real  Ratio 
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Figure  83  -  41  Net  Step  1/2  Rate  6  sec  Position  Update  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  84  -  41  Net  Step  1/2  Rate  6  sec  Position  Update  Cum  MS  and  MR  Interactions 
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Figure  85  -  41  Net  Step  1/2  Rate  6  sec  Position  Update  Cum  E,  R,  and  E+R 
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Figure  86  -  41  Net  Step  1/2  Rate  6  sec  Position  Update  60  sec  TMA(MS)  and  TMA(MR) 


60  sec  TMA(E)  and  TMA(R) 
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Figure  87  -  41  Net  Step  1/2  Rate  6  sec  Position  Update  60  sec  TMA(E)  and  TMA(R) 
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B.8  41  Net  Constant,  V2  Rate,  and  Position  Updates  every  10  sec  (Main 
PC) 
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Figure  89  -  41  Net  Constant  1/2  Rate  Ref  Net  Latency 
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Figure  90  -  41  Net  Constant  1/2  Rate  Cum  MS  and  MR  Interactions 


Cumulative  Expected  (E),  Relays  (R)  and  E+R 
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Figure  91  -  41  Net  Constant  1/2  Rate  Cum  E,  R,  and  E+R 
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Figure  92  -  41  Net  Constant  1/2  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  93  -  41  Net  Constant  1/2  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.9  10  Hour  Run,  41  Net  Constant,  V2  Rate,  and  Position  Updates 
every  10  sec  (Main  PC) 
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Figure  95  -  10  Hour  Run  41  Net  1/2  Rate  Ref  Net  Latency 
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Cumulative  MSG_SEND(MS)  and  MSG_RCVD(MR)  Interactions 
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Figure  96  -  10  Hour  Run  41  Net  1/2  Rate  Cum  MS  and  MR  Interactions 
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Figure  97  -  10  Hour  Run  41  Net  1/2  Rate  Cum  E,  R,  and  E+R 
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Figure  98-10  Hour  Run  41  Net  1/2  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  99  -  10  Hour  Run  41  Net  1/2  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.10  41  Net  Step,  V4  Rate,  and  Position  Updates  every  10  sec  (Laptop) 
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Figure  100  -  Laptop  Run  41  Net  Step  1/4  Rate  Sim/Real  Ratio 
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Figure  101  -  Laptop  Run  41  Net  Step  1/4  Rate  Ref  Net  Latency 


90 


1800 

1600 

1400 

1200 

1000 

800 

600 

400 

200 

0 


Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  102  -  Laptop  Run  41  Net  Step  1/4  Rate  Cum  MS  and  MR  Interactions 
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Figure  103  -  Laptop  Run  41  Net  Step  1/4  Rate  Cum  E,  R,  E+R 
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60  sec  TMA(MS)  and  TMA(MR) 
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Figure  104  -  Laptop  Run  41  Net  Step  1/4  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  105  -  Laptop  Run  41  Net  Step  1/4  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.11  41  Net  Step,  V2  Rate,  and  Position  Updates  every  10  sec  (Laptop) 
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Figure  106  -  Laptop  Run  41  Net  Step  1/2  Rate  Sim/Real  Ratio 
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Figure  107  -  Laptop  Run  41  Net  Step  1/2  Rate  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 


Figure  108  -  Laptop  Run  41  Net  Step  1/2  Rate  Cum  MS  and  MR  Interactions 
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Figure  109  -  Laptop  Run  41  Net  Step  1/2  Rate  Cum  E,  R,  and  E+R 
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Figure  110  -  Laptop  Run  41  Net  Step  1/2  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  111  -  Laptop  Run  41  Net  Step  1/2  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.12  41  Net  Step,  V2  Rate,  and  Position  Updates  every  10  sec  (Dif.  PC) 
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Figure  112  -  Dif.  PC  Run  41  Net  Step  1/2  Rate  Sim/Real  Ratio 
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Figure  113  -  Dif.  PC  Run  41  Net  Step  1/2  Rate  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 


Figure  114  -  Dif.  PC  Run  41  Net  Step  1/2  Rate  Cum  MS  and  MR  Interactions 
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Figure  115  -  Dif.  PC  Run  41  Net  Step  1/2  Rate  Cum  E,  R,  E+R 
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60  sec  TMA(MS)  and  TMA(MR) 
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Figure  116  -  Dif.  PC  Run  41  Net  Step  1/2  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  117  -  Dif.  PC  Run  41  Net  Step  1/2  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.13  41  Net  Step,  Full  Rate,  and  Position  Updates  every  10  sec  (Dif. 
PC) 
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Figure  118  -  Dif.  PC  Run  41  Net  Step  Full  Rate  Sim/Real  Ratio 
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Figure  119  -  Dif.  PC  Run  41  Net  Step  Full  Rate  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  120  -  Dif.  PC  Run  41  Net  Step  Full  Rate  Cum  MS  and  MR  Interactions 
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Figure  121  -  Dif.  PC  Run  41  Net  Step  Full  Rate  Cum  E,  R,  E+R 


100 


60  sec  TMA(MS)  and  TMA(MR) 


o 


500 


1000  1500  2000  2500  3000  3500 


File:  MQN_050720_17280203.xls 


Wall  Clock  (sec) 


Figure  122  -  Dif.  PC  Run  41  Net  Step  Full  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  123  -  Dif.  PC  Run  41  Net  Step  Full  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.14  81  Net  Step,  V2  Rate,  and  Position  Updates  every  10  sec  (Main 
PC) 
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Figure  124  -  81  Net  Step  1/2  Rate  Sim/Real  Ratio 
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Figure  125  -  81  Net  Step  1/2  Rate  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  126  -  81  Net  Step  1/2  Rate  Cum  MS  and  MR  Interactions 
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Figure  127  -  81  Net  Step  1/2  Rate  Cum  E,  R,  and  E+R 
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Case:  81  Net  Step  1/2  Rate 
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Figure  128  -  81  Net  Step  1/2  Rate  60  sec  TMA(MS)  and  TMA(MR) 
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Figure  129  -  81  Net  Step  1/2  Rate  60  sec  TMA(E)  and  TMA(R) 


104 


B.15  81  Net  Step,  Full  Rate,  and  Position  Updates  every  10  sec  (Main 
PC) 
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Figure  130  -  81  Net  Step  Full  Rate  Sim/Real  Ratio 
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Figure  131  -  81  Net  Step  Full  Rate  Ref  Net  Latency 
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Cumluative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  132  -  81  Net  Step  Full  Rate  Cum  MS  and  MR  Interactions 
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Figure  133  -  81  Net  Step  Full  Rate  Cum  E,  R,  and  E+R 
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Figure  134  -  81  Net  Step  Full  Rate  60  sec  TMA(MS)  and  TMA(MR) 


Figure  135  -  81  Net  Step  Full  Rate  60  sec  TMA(E)  and  TMA(R) 
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B.16  81  Net  Step,  Full  Rate,  and  Position  Updates  every  14  sec  (Main  PC) 


Sim/Real  Ratio 


Figure  136  -  81  Net  Step  Full  Rate  14  sec  Position  Update  Sim/Real  Ratio 
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Figure  137  -  81  Net  Step  Full  Rate  14  sec  Position  Update  Ref  Net  Latency 
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Cumulative  MSG_SEND  (MS)  and  MSG_RCVD  (MR)  Interactions 
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Figure  138  -  81  Net  Step  Full  Rate  14  sec  Position  Update  Cum  MS  and  MR  Interactions 
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Figure  139-  81  Net  Step  Full  Rate  14  sec  Position  Update  Cum  E,  R,  and  E+R 
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Figure  140  -  81  Net  Step  Full  Rate  14  sec  Position  Update  60  sec  TMA(MS)  and  TMA(MR) 


60  sec  TMA(E)  and  TMA(R) 


Figure  141  -  81  Net  Step  Full  Rate  14  sec  Position  Update  60  Sec  TMA(E)  and  TMA(R) 
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B.17  81  Net  Step,  Full  Rate,  and  Position  Updates  every  6  sec  (Main 
PC) 


Sim/Real  Ratio 


Figure  142  -  81  Net  Step  Full  Rate  6  sec  Position  Update  Sim/Real  Ratio 
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Figure  143-  81  Net  Step  Full  Rate  6  sec  Position  Update  Ref  Net  Latency 
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Figure  144-  81  Net  Step  Full  Rate  6  sec  Position  Update  Cum  MS  and  MR  Interactions 


Cumulative  Expected  (E),  Relays  (R)  and  E+R 
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Figure  145  -  81  Net  Step  Full  Rate  6  sec  Position  Update  Cum  E,  R,  E+R 
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Figure  146  -  81  Net  Step  Full  Rate  6  sec  Position  Update  60  sec  TMA(MS)  and  TMA(MR) 


60  sec  TMA(E)  and  TMA(R) 


Number  of  Platforms: 

60 

180  n 

Platform  Update: 

ALL 

Update  Type: 

Bulk 

160  - 

Update  Rate  (sec): 

6 

o 

o 

03 

U) 

ro 

CD 

> 

< 


File 


0  500 

MON  050719  19135701.xls 
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Figure  147  -  81  Net  Step  Full  Rate  6  sec  Position  Update  60  sec  TMA(E)  and  TMA(R) 
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